Chapter W11. CGI Programming
 
Goals for this chapter: rpm packages covered in this chapter: 
  • httpd (apache)
  • perl
  • perl-modules
 

Deliver yesterday, code today, think tomorrow
-- Anonymous

The Common Gateway Interface

The Common Gateway Interface, CGI, is one of the solutions used on the Web to interact with Web servers,  its Database and other servers.

For example with the Common Gateway Interface is possible to enter credit card information, customer data, search engine and other.

Generally the programming language used to interact between the client and the remote Web Server is PERL (See Chapter 12. Programming with PERL). Is also possible write programs in Visual Basic, C, Tcl and C Shell and other languages.

The CGI files are directly connected with the Apache configuration files and directories. One mode to understand the Gateway Interface is create binaries in the Web server and the client run these binaries. The programming language to create these binaries generally is PERL.

For example, we can browse Linux Manual pages using CGI and PERL.
 
 

http://www.FTLinuxCourse.com/FTLC_Complete/FTLinuxCourse/en/indexs.html

The question is that PERL (or any other programming language) may run a Linux executable and using the programming language.

Before to start to test your CGI scripts be sure that your "/etc/hosts" includes:

[root@ftosx1 /root]# more /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1               localhost.localdomain localhost
192.168.1.93            ftosx1.redhat.com ftosx1
[root@ftosx1 /root]#

Test that your apache daemon is running, with the command: "ps ax | grep httpd".

Now, you can run netscape with the command:
 
 

Once you see the previous page, your httpd is working and you can start to write CGI scripts and test it.

For example now you can run a CGI script like "man2html"
 

man2html input page

How we can see the http request here is:

http://ftosx1.futuretg.com/cgi-bin/man2html/usr/man/man1/gcc.1.gz

The man2html may be a binary, like

[root@ftosx1 cgi-bin]# pwd
/home/httpd/cgi-bin
[root@ftosx1 cgi-bin]# file man2html
man2html: ELF 32-bit LSB executable, Intel 80386, version 1, dynamically linked (uses shared libs), stripped
[root@ftosx1 cgi-bin]#

If you run this binary, the output will be an HTML file, that appears in the first page:

[root@ftosx1 cgi-bin]# ./man2html
Content-type: text/html

<HTML>
<HEAD>
<TITLE>Manual Pages - Main Contents</TITLE>
<!-- Changed by: Michael Hamilton,  4-May-1996 -->
</HEAD>
<BODY>
<H1>Manual Pages - Main Contents</H1>
<HR>

<H2>Name and Section lookup</H2>
<ISINDEX>

You can enter a program name, the section, an extra
directory (using -M) or a full name.  For example:
<UL>
   <LI><TT>find</TT>
   <LI><TT>find 1</TT>
   <LI><TT>-M /usr/local/man find</TT>
   <LI><TT>/local/gcc/man/man1/gperf.1</TT>
</UL>

<HR>

<H2>Index of pages by name and description</H2>
Sections:
  <A HREF="http:/cgi-bin/manwhatis?1">1. User Commands</A>;
  <A HREF="http:/cgi-bin/manwhatis?2">2. System Calls</A>;
  <A HREF="http:/cgi-bin/manwhatis?3">3. Library Functions</A>;
  <A HREF="http:/cgi-bin/manwhatis?4">4. Special Files</A>;
  <A HREF="http:/cgi-bin/manwhatis?5">5. File Formats</A>;
  <A HREF="http:/cgi-bin/manwhatis?6">6. Games</A>;
  <A HREF="http:/cgi-bin/manwhatis?7">7. Miscellany</A>;
  <A HREF="http:/cgi-bin/manwhatis?8">8. Administration and Privileged Commands</A>.
  <A HREF="http:/cgi-bin/manwhatis?9">9. Kernel Reference Guide</A>.
<P>
<HR>
<H2>Index of pages by name only</H2>
  <A HREF="http:/cgi-bin/mansec?1">1. User Commands</A>;
  <A HREF="http:/cgi-bin/mansec?2">2. System Calls</A>;
  <A HREF="http:/cgi-bin/mansec?3">3. Library Functions</A>;
  <A HREF="http:/cgi-bin/mansec?4">4. Special Files</A>;
  <A HREF="http:/cgi-bin/mansec?5">5. File Formats</A>;
  <A HREF="http:/cgi-bin/mansec?6">6. Games</A>;
  <A HREF="http:/cgi-bin/mansec?7">7. Miscellany</A>;
  <A HREF="http:/cgi-bin/mansec?8">8. Administration and Privileged Commands</A>;
  <A HREF="http:/cgi-bin/mansec?9">9. Kernel Reference Guide</A>;
  <A HREF="http:/cgi-bin/mansec?all">All Sections</A>.

<P>
<hr>

<H2>Manual Pages full text search</H2>

<A HREF="http:/cgi-bin/mansearch">
Search the full text of the Manual Pages.
</A>

<P>
<HR>

The original <I>VH-Man2html</I> was written by
<A HREF="http://wsinwp01.win.tue.nl:1234/index.html">
Richard Verhoeven</A>.
<I>VH-Man2html-1.5</I> was enhanced and customised for Linux by
<A HREF="mailto:michael@actrix.gen.nz">
Michael Hamilton</A>.
</body>
</HTML>[root@ftosx1 cgi-bin]#

In this mode, CGI allow to run, Linux (or any binary) from the HTML pages. For example, if will run the Web server in Mac or MS Windows, you will run it on a MS

Apache Configuration

Before to present CGI examples and test, is important to understand that CGI is an Apache component. This means that CGI is included in Apache daemon.

By default, generally nothing is necessary to configure and everything works without any effort, but is important to know what lines regards CGI in the Apache configuration for the local and virtual Web servers.

In "/etc/httpd/conf/httpd.conf", we will found the following lines:

#
# ScriptAlias: This controls which directories contain server scripts.
# ScriptAliases are essentially the same as Aliases, except that
# documents in the realname directory are treated as applications and
# run by the server when requested rather than as documents sent to the client.
# The same rules about trailing "/" apply to ScriptAlias directives as to
# Alias.
#
ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"

#
# "/var/www/cgi-bin" should be changed to whatever your ScriptAliased
# CGI directory exists, if you have that configured.
#
<Directory "/var/www/cgi-bin">
    AllowOverride None
    Options ExecCGI
    Order allow,deny
    Allow from all
</Directory>
 

The last lines fix the path and the behavior.

You can add or change the directories if you prefer.

Another important file to list is: /etc/httpd/logs/

This is a classical view on html errors.

[Sun Jun 10 04:02:42 2001] [error] [client 209.73.164.130] File does not exist: /var/www/html/robots.txt
[Sun Jun 10 04:04:15 2001] [error] [client 212.135.130.130] File does not exist: /var/www/html/FTOSX/FTKDE/index.html../index.html
[Sun Jun 10 04:11:49 2001] [error] [client 209.73.164.130] File does not exist: /var/www/html/robots.txt
[Sun Jun 10 04:19:21 2001] [error] [client 216.35.116.61] File does not exist: /var/www/html/ftyellow/Airplanes.html
[Sun Jun 10 04:45:33 2001] [error] [client 216.35.116.54] File does not exist: /var/www/html/FTYellowPages/.tj.html
...
 

First example.

Now we will explain a primary example using HTML and CGI to print Linux output inside an HTML page.

... but how works CGI?

The client send a request inside a form (manual page) ... the server receive the request and send back the answer to the client.
 
 

This means that using an HTML file the client imput the request. Then, the request is handled by a PERL script.

Suppose that you want to print the date, the cal or who the users connected to your system.

Then, you can use the CGI.

You need two components, a simple HTML file and a PERL script.

We list here the HTML file and the PERL script
 
 

<!doctyp[root@ftosx1 cgi-bin]# ./mydate.pl
Content-type: text/plain

Thu Jun  7 11:17:33 /etc/localtime 2001
[root@ftosx1 cgi-bin]#e html public "-//w3c/[root@ftosx1 cgi-bin]# ./mydate.pl
Content-type: text/plain

Thu Jun  7 11:17:33 /etc/localtime 2001
[root@ftosx1 cgi-bin]#/dtd html 4.0 transitional//en">
<html>
<head>
   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   <meta name="GENERATOR" content="Mozilla/4.77 [en] (X11; U; Linux 2.4.5 i686) [Netscape]">
</head>
<body>
Simple form
<p>
<hr WIDTH="100%">
<FORM ACTION="/cgi-bin/mydate.pl" METHOD="GET">
Command: <INPUT TYPE="text" NAME="command" SIZE=40>
<P>
<INPUT TYPE="submit" VALUE="Submit form">
<INPUT TYPE="reset" VALUE="Clear form">
</FORM>
<br>Enter your command:
<br>&nbsp;
<br>&nbsp;
<p>
<hr WIDTH="100%">
</body>
</html>

This is the simple HTML file, that will includes a form to interact with the PERL Script This is the simple.html file

Note this file is located in "/home/httpd/html"

#!/usr/bin/perl

$request_method = $ENV{'REQUEST_METHOD'};

if ($request_method eq "GET" ) {
        $form_info = $ENV{'QUERY_STRING'};
} else {
        $size_of_form_information = $ENV{'CONTENT_LENGTH'};
        read (STDIN, $form_info, $size_of_form_information );
}

($field_name, $command) = split (/=/, $form_info);

print "Content-type: text/plain", "\n\n";

if ($command eq "cal") {
        print `/usr/bin/cal`;
} elsif ($command eq "finger") {
        print `/usr/bin/finger`;
} else {
        print `/bin/date`;
}
exit (0);

[root@ftosx1 cgi-bin]# ./mydate.pl
Content-type: text/plain

Thu Jun  7 11:17:33 /etc/localtime 2001
[root@ftosx1 cgi-bin]#

Note this file is located in "/home/httpd/cgi-bin"

Note the final URL is:

http://localhost/cgi-bin/mydate.pl?command=finger

Similar URL will appear if the client choose: date or cal command.

CGI uses basically two methods: GET and POST to send data:

We can change the Method from GET to POST. Is necessary to change only one line of code.
 
<!doctyp[root@ftosx1 cgi-bin]# ./mydate.pl
Content-type: text/plain

Thu Jun  7 11:17:33 /etc/localtime 2001
[root@ftosx1 cgi-bin]#e html public "-//w3c/[root@ftosx1 cgi-bin]# ./mydate.pl
Content-type: text/plain

Thu Jun  7 11:17:33 /etc/localtime 2001
[root@ftosx1 cgi-bin]#/dtd html 4.0 transitional//en">
<html>
<head>
   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   <meta name="GENERATOR" content="Mozilla/4.77 [en] (X11; U; Linux 2.4.5 i686) [Netscape]">
</head>
<body>
Simple form
<p>
<hr WIDTH="100%">
<FORM ACTION="/cgi-bin/mydate.pl" METHOD="POST">
Command: <INPUT TYPE="text" NAME="command" SIZE=40>
<P>
<INPUT TYPE="submit" VALUE="Submit form">
<INPUT TYPE="reset" VALUE="Clear form">
</FORM>
<br>Enter your command:
<br>&nbsp;
<br>&nbsp;
<p>
<hr WIDTH="100%">
</body>
</html>

This is the simple HTML file, that will includes a form to interact with the PERL Script This is the simple.html file

Note this file is located in "/home/httpd/html"

#!/usr/bin/perl

$request_method = $ENV{'REQUEST_METHOD'};

if ($request_method eq "GET" ) {
        $form_info = $ENV{'QUERY_STRING'};
} else {
        $size_of_form_information = $ENV{'CONTENT_LENGTH'};
        read (STDIN, $form_info, $size_of_form_information );
}

($field_name, $command) = split (/=/, $form_info);

print "Content-type: text/plain", "\n\n";

if ($command eq "cal") {
        print `/usr/bin/cal`;
} elsif ($command eq "finger") {
        print `/usr/bin/finger`;
} else {
        print `/bin/date`;
}
exit (0);

If we run locally the script 

[root@ftosx1 cgi-bin]# ./mydate.pl
Content-type: text/plain

Thu Jun  7 11:17:33 /etc/localtime 2001
[root@ftosx1 cgi-bin]#

Note this file is located in "/home/httpd/cgi-bin"

Therefore using this method we can print inside an HTML page the output of any Linux program including your own programs!

Before to continue with new examples and learn how to interact forms with CGI scripts, we need to know the CGI Enviroment variables.

A classical script for this task is the following:
 

#!/usr/bin/perl

%list = ('SERVER_SOFTWARE',   'The server software is: ',
         'SERVER_NAME',       'The server hostname, DNS alias, or IP address is: ',
         'GATEWAY_INTERFACE', 'The CGI specification revision is: ',
         'SERVER_PROTOCOL',   'The name and revision of info protocol is: ',
         'SERVER_PORT',       'The port number for the server is: ',
         'REQUEST_METHOD',    'The info request method is: ',
         'PATH_INFO',         'The extra path info is: ',
         'PATH_TRANSLATED',   'The translated PATH_INFO is: ',
         'DOCUMENT_ROOT',     'The server document root directory is: ',
         'SCRIPT_NAME',       'The script name is: ',
         'QUERY_STRING',      'The query string is (FORM GET): ',
         'REMOTE_HOST',       'The hostname making the request is: ',
         'REMOTE_ADDR',       'The IP address of the remote host is: ',
         'AUTH_TYPE',         'The authentication method is: ',
         'REMOTE_USER',       'The authenticated user is: ',
         'REMOTE_IDENT',      'The remote user is (RFC 931): ',
         'CONTENT_TYPE',      'The content type of the data is (POST, PUT): ',
         'CONTENT_LENGTH',    'The length of the content is: ',
         'HTTP_ACCEPT',       'The MIME types that the client will accept are: ',
         'HTTP_USER_AGENT',   'The browser of the client is: ',
         'HTTP_REFERER',      'The URL of the referer is: ');

print "Content-type: text/html","\n\n";

print "<HTML>", "\n";
print  "<HEAD><TITLE>List of Environment Variables</TITLE></HEAD>", "\n";
print "<BODY>", "\n";
print "<H1>", "CGI Environment Variables", "</H1>", "<HR>", "\n";

while ( ($env_var, $info) = each %list ) {
    print $info, "<B>", $ENV{$env_var}, "</B>", "<BR>","\n";
}

print "<HR>", "\n";
print "</BODY>", "</HTML>", "\n";

exit (0);

The last example is not an HTML file that uses a PERL script, but a PERL script.

This show that we can use a single PERL script to obtain results.

We will complete this section teaching how to interact Forms with CGI

Forms and CGI

In the previous example we evaluate an input field.  Now, we will cover all the form components and how they interact with the Common Gatway Interface, CGI.

The mode to call CGI scripts inside forms is using the FORM ACTION tag, in the following mode:

<FORM ACTION="/cgi-bin/myscript.pl" METHOD="POST">

Like we explain before the language used may be any: Tcl, PERL, BASIC, and others.

Submit and Reset Buttons

We will start with a new example based on Submit and Reset buttons. In these sections we will use the examples in a debug mode presenting the input results.

Generally, any FORM includes these two input buttons:

Inside a FORM is possible to apply names and values.

For example we can have forms with two or more buttons:

<INPUT TYPE="submit" NAME="option" VALUE="First button">
<INPUT TYPE="submit" NAME="option" VALUE="Second button">

Now, when the user hit the first submit button will have option=First button.
 

Text and Password Fields
 
 
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   <meta name="GENERATOR" content="Mozilla/4.77 [en] (X11; U; Linux 2.4.5 i686) [Netscape]">
</head>
<body>
<form ACTION="/cgi-bin/passwd.pl" METHOD="POST">
<P>Name:
<input TYPE="text" NAME="user" SIZE=40 MAXLENGTH=40 VALUE="John Smith">
<P>
<P>Password:
   <INPUT TYPE="password" NAME="password" SIZE="12" VALUE="treasure">
   <INPUT TYPE="hidden" NAME="DefaultPass" VALUE="treasure">
  <P>
<br><input TYPE="submit" VALUE="Log in"><input TYPE="reset" VALUE="Clear">
</FORM>
</body>
</html>
The HTML form, submit.html Input page

#!/usr/bin/perl

$request_method = $ENV{'REQUEST_METHOD'};

$size_of_form_information = $ENV{'CONTENT_LENGTH'};
read (STDIN, $form_info, $size_of_form_information );

($field_name, $command) = split (/=/, $form_info);

print "Content-type: text/plain", "\n\n";
 

print "$form_info\n";
print "$command\n";

exit (0);

The passwd.pl in the cgi-bin directory The data handled by the CGI

Radio Buttons and Checkboxes

Now, we will cover radio and check buttons:
 

<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   <meta name="GENERATOR" content="Mozilla/4.77 [en] (X11; U; Linux 2.4.5 i686) [Netscape]">
</head>
<body>
<form ACTION="/cgi-bin/radio.pl" METHOD="POST">

  <P>Specify your car preferences (check all that apply):</P>
  <P><INPUT TYPE="checkbox" NAME="german_cars" CHECKED> German Cars
   <BR><INPUT TYPE="checkbox" NAME="american_cars" > American Cars
   <BR><INPUT TYPE="checkbox" NAME="french_cars" > French Cars
   <BR><INPUT TYPE="checkbox" NAME="italian_cars" > Italian Cars

<P>Choose a type for your car:
<UL>
  <INPUT TYPE="radio" NAME="category" VALUE="family"> Family <BR>
  <INPUT TYPE="radio" NAME="category" VALUE="coupe"> Coupe <BR>
  <INPUT TYPE="radio" NAME="category" VALUE="cabriolet" CHECKED> Cabriolet
</UL>

<br><input TYPE="submit" VALUE="Submit"><input TYPE="reset" VALUE="Clear">
</FORM>
</body>
</html>
 

The source for the HTML form
HTML form including radio and checkboxes

#!/usr/bin/perl

$request_method = $ENV{'REQUEST_METHOD'};

$size_of_form_information = $ENV{'CONTENT_LENGTH'};
read (STDIN, $form_info, $size_of_form_information );

($field_name, $command) = split (/&/, $form_info);

print "Content-type: text/plain", "\n\n";
 

print "$form_info\n";
print "$field_name\n";
print "$command\n";

exit (0);

The source for the PERL script
The output handled by the CGI.

Menus and Scrolled Lists
 
 
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   <meta name="GENERATOR" content="Mozilla/4.77 [en] (X11; U; Linux 2.4.5 i686) [Netscape]">
</head>
<body>
<form ACTION="/cgi-bin/menu.pl" METHOD="POST">
 
 

<B>Shipping method:</B><BR>
<SELECT name="ups" SIZE=1 >
        <OPTION> UPS
        <OPTION SELECTED> Mail
        <OPTION> FedEx Overnight
</SELECT>
<br><input TYPE="submit" VALUE="Submit"><input TYPE="reset" VALUE="Clear">
</FORM>
</body>
</html>

The source for the HTML form

#!/usr/bin/perl

$request_method = $ENV{'REQUEST_METHOD'};

$size_of_form_information = $ENV{'CONTENT_LENGTH'};
read (STDIN, $form_info, $size_of_form_information );

($field_1, $field_2) = split (/=/, $form_info);

print "Content-type: text/plain", "\n\n";
 

print "$form_info\n";
print "$field_1\n";
print "$field_2\n";

exit (0);

The source for the PERL form
The output handled by the CGI.

Multiline text field

In this example we will use Multiline text field.
 

<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   <meta name="GENERATOR" content="Mozilla/4.77 [en] (X11; U; Linux 2.4.5 i686) [Netscape]">
</head>
<body>
<form ACTION="/cgi-bin/textarea.pl" METHOD="POST">
 

<B>Description:</B>
<BR><TEXTAREA NAME="item_description" ROWS="6" COLS="55">
This is the first line.
This is the second line.

This is the last line.
</TEXTAREA>
<br><input TYPE="submit" VALUE="Submit"><input TYPE="reset" VALUE="Clear">
</FORM>
</body>
</html>

The HTML form
The HTML form
#!/usr/bin/perl

$request_method = $ENV{'REQUEST_METHOD'};

$size_of_form_information = $ENV{'CONTENT_LENGTH'};
read (STDIN, $form_info, $size_of_form_information );

($field_1, $field_2) = split (/&/, $form_info);
#($field_3, field_4, field_5) = split (/&/, $form_info);

print "Content-type: text/plain", "\n\n";
 

print "$form_info\n";
#print `/usr/sbin/sendmail -v gorlando@futuretg.com < $form_info`;

exit (0);

 

The PERL source 
The output handled by the CGI.

Plotting with CGI

The CGI may be used also to plot graphics. For example the PGPLOT package allows to use  a simple PERL commands to create graphics like the following:
 
 

These type of graphics are used to create web site statistics, NASDAQ quotes on real time and other useful CGI graphical figures.
 

Web Site Statistics

On the Web there are thousands of programs to create web site statistics. We present here the Analog analizer and some views.
 

Setup the statistics Present the Web site statistics

Exercises

  1. Modify the first example using a form with three buttons, one for each command.
  2. Modify the textarea example to send an email.
Tests
  1. Write a PERL script that using CGI files prints "Hello Web!".
  2. Write a CGI script that input your birthday and prints out the day of the week where you born
  3. Where may be listed a probable CGI error ?
  4. Is possible to write a CGI script in C-shell ?
  5. What is the FORM TAG to call the SCRIPT: myscript.pl ?
  6. List the two methods to send data to the CGI server ?
  7. Is possible to print the output of the command: ls , inside an HTML page using CGI scripts ?
  8. Where are located the CGI scripts ?
  9. Where is located the file passwd.html, when you run the command: http://localhost/passwd.html ?
  10. Is possible to have PERL scripts that create HTML output ?
  11. Is possible to create graphics with CGI ?
Read the answers to the exercises.
 

Check the Interactive Exam Cram WebMaster: Try the interactive cram ...
 
 

Internet Resources for this Chapter.