CGI-Bin Explained
The cgi-bin directory (short for “Common Gateway Interface binaries”) was a special folder on web servers where executable programs lived. When a browser requested a URL like /cgi-bin/formmail.pl, the server ran that program and returned its output as a web page. Every shared hosting provider from 1994 to the mid-2000s had a cgi-bin directory — it was the only way to add dynamic functionality to a website.
In Brief
The cgi-bin was a server directory for executable scripts, required by every 1990s hosting provider. Files inside were executed rather than served — the foundation of all dynamic web content before PHP.
What cgi-bin Actually Was
The cgi-bin directory was not a piece of software. It was a directory on the filesystem — a folder, no different from any other folder on a Unix server — that the web server treated with special rules. While the rest of the document root (/var/www/html/ or /home/user/public_html/) served files as static content, files inside cgi-bin were executed. A request to /images/logo.gif returned the raw bytes of the image file. A request to /cgi-bin/counter.pl ran the Perl script and returned whatever text it printed.
The mechanism that made this work in Apache was the ScriptAlias directive:
ScriptAlias /cgi-bin/ /usr/local/apache/cgi-bin/This single line told Apache two things simultaneously. First, it mapped the URL path /cgi-bin/ to the filesystem directory /usr/local/apache/cgi-bin/. Second, it declared that every file inside that directory was an executable program. Without ScriptAlias, Apache would serve the Perl source code as plain text — visitors would see #!/usr/bin/perl and the raw code instead of the script’s output.
The security rationale was straightforward: executable code must be isolated from static content. In the early 1990s, web servers had limited access control mechanisms. Putting all executables in one directory made it possible to apply strict permissions, logging, and execution policies to a single location. The web server administrator could lock down cgi-bin with specific user ownership, group restrictions, and filesystem permissions while leaving the rest of the document root open for HTML, images, and CSS files. This separation was considered best practice and was enforced by nearly every shared hosting provider.
On most Unix systems, the cgi-bin directory had permissions set to drwxr-xr-x (755) and was owned by root or the web server user. The directory path varied by operating system and distribution:
| System | Typical cgi-bin Path |
|---|---|
| Debian / Ubuntu | /usr/lib/cgi-bin/ |
| Red Hat / CentOS | /var/www/cgi-bin/ |
| Apache (compiled from source) | /usr/local/apache2/cgi-bin/ |
| Shared hosting (cPanel) | ~/public_html/cgi-bin/ |
| NCSA HTTPd (original) | /usr/local/etc/httpd/cgi-bin/ |
How cgi-bin Worked (Step by Step)
When a user clicked a link or submitted a form that pointed to a cgi-bin URL, the following sequence occurred on the server. This process happened identically for every single request — there was no caching, no connection pooling, no persistence between requests.
Browser sends request
↓
GET /cgi-bin/counter.pl HTTP/1.0
↓
Apache receives request, matches ScriptAlias /cgi-bin/
↓
Apache calls fork() — creates a new OS process
↓
New process executes: /usr/bin/perl /usr/local/apache/cgi-bin/counter.pl
↓
Perl interpreter starts, parses counter.pl, loads modules
↓
counter.pl reads count.dat file, increments the number, writes it back
↓
counter.pl prints: Content-type: text/html\n\n
counter.pl prints: <html><body>You are visitor 14,392</body></html>
↓
Apache captures stdout, sends it as HTTP response
↓
Process terminates — memory freed, file handles closed
↓
Browser renders the pageThe critical detail is step three: fork(). Every request to a cgi-bin script created an entirely new operating system process. The Perl interpreter was loaded from disk into memory. All modules referenced by use statements were located, read, and compiled. The script executed, printed its output, and the process was destroyed. If ten users hit the same counter script simultaneously, ten separate Perl interpreters ran in parallel, each consuming 5–15 MB of RAM.
For small sites with a few hundred visitors per day, this was perfectly adequate. The process startup took 50–200 milliseconds — imperceptible to users on dial-up connections where the network latency alone was several hundred milliseconds. But as the web grew, the fork-per-request model became the bottleneck that ultimately drove the industry to develop alternatives like mod_perl, FastCGI, and mod_php.
The chmod 755 Ritual
chmod 755 — owner can read/write/execute, everyone else can read/execute. Every webmaster of the CGI era typed this command. Getting it wrong meant a 500 Internal Server Error.
Every webmaster who deployed CGI scripts in the 1990s went through the same ritual, and every one of them got it wrong at least once. The process was deceptively simple but filled with traps that produced the most feared message on the early web: 500 Internal Server Error.
The Deployment Steps
- Open your FTP client — WS_FTP, CuteFTP, or the command-line
ftputility. Connect to your shared hosting account. - Switch to ASCII transfer mode — this was the single most important step that beginners missed. FTP has two transfer modes: binary (transfers bytes unchanged) and ASCII (translates line endings between systems). Perl scripts written on Windows used
\r\n(carriage return + line feed) for line endings. Unix servers expected\n(line feed only). Uploading in binary mode preserved the\rcharacters, which caused the shebang line (#!/usr/bin/perl\r) to be interpreted as a request to run a program calledperl\r— which did not exist. - Upload the script to cgi-bin — navigate to
/cgi-bin/or/public_html/cgi-bin/and transfer the.plfile. - Set permissions:
chmod 755— this was done either through the FTP client’s permissions dialog or via SSH with the commandchmod 755 script.pl. - Test in the browser — navigate to
http://yourdomain.com/cgi-bin/script.pland pray.
What 755 Actually Means
The number 755 is an octal (base-8) representation of Unix file permissions. Each digit represents permissions for a different class of users:
| Digit | Who | Value | Permissions | Meaning |
|---|---|---|---|---|
| 7 | Owner | 4+2+1 | rwx |
Read, write, and execute |
| 5 | Group | 4+0+1 | r-x |
Read and execute (no write) |
| 5 | Others | 4+0+1 | r-x |
Read and execute (no write) |
The web server process (typically running as user nobody, www-data, or apache) fell into the “others” category. It needed both read permission (to read the script’s source code) and execute permission (to run it). Without execute permission, Apache returned a 403 Forbidden error. Without read permission, the Perl interpreter could not load the file.
The Three Most Common Mistakes
Binary Upload (CRLF)
Uploading in binary mode instead of ASCII preserved Windows line endings (\r\n). The shebang line became #!/usr/bin/perl\r, and the server tried to execute a non-existent program. The error log showed: exec format error or No such file or directory.
Wrong Permissions
Setting permissions to 644 instead of 755 meant the file was readable but not executable. The server returned 403 Forbidden. Some beginners tried 777 (read/write/execute for everyone), which worked but was a severe security hole — any user on the shared server could modify the script.
Wrong Shebang Path
The first line of every CGI script had to specify the path to the interpreter: #!/usr/bin/perl. But Perl was not always at /usr/bin/perl. On some systems it was /usr/local/bin/perl or /opt/perl/bin/perl. The wrong path produced a cryptic 500 error with no useful output.
These three issues accounted for the vast majority of “my CGI script doesn’t work” support requests on every web hosting provider throughout the late 1990s. Hosting companies wrote extensive knowledge base articles about them. Matt Wright’s scripts included a detailed troubleshooting section in every README file specifically addressing these problems.
NCSA HTTPd: Where cgi-bin Was Born
The cgi-bin convention did not emerge from a standards committee or an industry consortium. It was created by Rob McCool, a programmer at the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign. In November 1993, McCool released NCSA HTTPd, which quickly became the most popular web server in the world.
NCSA HTTPd introduced the concept of a dedicated directory for executable scripts. McCool’s design was influenced by the Unix filesystem convention of storing executables in /bin, /usr/bin, and /usr/local/bin directories. The name cgi-bin followed this pattern: it was the “bin” directory for CGI programs. The NCSA HTTPd configuration file used a directive to map the URL path to the filesystem:
# NCSA HTTPd configuration (circa 1993)
Exec /cgi-bin/* /usr/local/etc/httpd/cgi-bin/*This Exec directive was the precursor to Apache’s ScriptAlias. It told the server that any URL matching /cgi-bin/* should be executed as a program, with the filesystem location of the scripts specified separately. McCool also defined the environment variables (QUERY_STRING, REMOTE_ADDR, REQUEST_METHOD, and others) that the server would pass to CGI scripts — the core of what would later become the CGI specification formalized in RFC 3875.
By mid-1994, NCSA HTTPd ran on roughly 90% of all web servers on the internet. But McCool left NCSA in late 1994, and the project lost momentum. In February 1995, a group of eight webmasters who had been independently patching NCSA HTTPd to fix bugs and add features decided to combine their work. They called their project Apache — popularly said to be “a patchy server,” though the Apache Foundation has offered alternative explanations for the name. Apache inherited NCSA HTTPd’s cgi-bin convention, refined it with the ScriptAlias directive, and carried it into the next two decades of web development.
What You’d Find in cgi-bin
If you could have looked inside the cgi-bin directory of a typical shared hosting account in 1997, you would have found a remarkably similar collection of files across millions of websites. The same scripts appeared over and over, because the same handful of free Perl programs solved the problems that every webmaster faced.
The Standard Collection
| Script | Purpose | Author |
|---|---|---|
| formmail.pl | Processed HTML form submissions and sent them as email. By far the most widely deployed CGI script ever written. Appeared on millions of websites. | Matt Wright |
| guestbook.pl | Let visitors sign a guestbook by submitting their name, email, and a message. Entries were appended to a flat HTML file. | Matt Wright |
| counter.pl | Hit counter that incremented a number stored in a .dat file on every page view. Often displayed as graphical digit images. |
Matt Wright / various |
| search.pl | Flat-file search engine that used grep to find matching terms in HTML files on the server. |
Matt Wright |
| wwwboard.pl | Threaded discussion forum that stored messages as individual HTML files with parent-child relationships. | Matt Wright |
| rand_link.pl | Redirected the visitor to a random URL from a predefined list. Used for “random link” buttons and webrings. | Matt Wright |
Matt Wright’s scripts, distributed from this very domain (worldwidemart.com), were the most popular CGI scripts of the era. They were free, well-documented, and easy for beginners to install. FormMail alone was running on an estimated 2–3 million websites by 2000.
Supporting Libraries
Alongside the scripts themselves, you would often find one or more Perl library files:
- cgi-lib.pl — Steven Brenner’s CGI library (1993), the first widely used Perl module for parsing CGI form data. Its
ReadParse()function became the standard way to read form input beforeCGI.pmexisted. - CGI.pm — Lincoln Stein’s comprehensive CGI module, included in the Perl core distribution from Perl 5.004 (1997) to Perl 5.20 (2014). It handled form parsing, cookie management, file uploads, and HTML generation.
CGI.pmreplacedcgi-lib.plas the standard approach by the late 1990s. - Sendmail path configuration — many scripts needed the path to the
sendmailbinary (typically/usr/sbin/sendmailor/usr/lib/sendmail) to send email. This path was hardcoded at the top of each script that sent mail.
The cgi-bin directory also frequently contained data files: count.dat for hit counters, guestbook.html for guestbook entries, .htaccess files for additional Apache configuration, and sometimes .log files generated by the scripts themselves. These data files needed to be writable by the web server process, which meant setting permissions to 666 (read/write for everyone) or 777 for directories — a constant source of security concern on shared hosting environments.
Why cgi-bin Disappeared
The cgi-bin directory did not disappear because of a single event. It faded away gradually as web technology made it unnecessary. The forces that killed cgi-bin were:
PHP Changed the Rules
When PHP emerged as an Apache module (mod_php), it fundamentally changed how server-side code was deployed. A PHP file could be placed anywhere in the document root — /about/index.php, /contact.php, /blog/post.php — and Apache would execute it automatically based on the .php file extension. There was no need for a special directory. No ScriptAlias. No chmod 755. The web server recognized PHP files by their extension and passed them to the embedded PHP interpreter.
This was revolutionary in its simplicity. A webmaster could mix HTML and PHP in the same file, place it alongside images and CSS files, and it just worked. The URL structure of a PHP-powered site looked clean: /contact.php instead of /cgi-bin/contact.pl. The /cgi-bin/ prefix — which had been a visible indicator of server-side processing — vanished from URLs.
mod_perl and mod_python Embedded the Interpreter
mod_perl (1996) and later mod_python loaded the Perl and Python interpreters directly into the Apache process. Scripts no longer needed to be in a special directory because they were not being forked as separate processes. Instead, the web server itself could execute Perl or Python code from any location, configured by SetHandler directives and location blocks. The performance was 10–100x better than traditional CGI because the interpreter loaded once and stayed in memory.
Shared Hosting Evolved
By the early 2000s, shared hosting control panels like cPanel (released 1996, widely adopted by 2001) and Plesk automated server configuration. These panels configured Apache to handle PHP by default, set up MySQL databases with a click, and provided web-based file managers. The cgi-bin directory still existed — cPanel created one in every new account — but most users never opened it. PHP was the default, and the hosting industry built its entire product around that assumption.
Security Concerns Accelerated the Shift
The cgi-bin directory became a well-known attack vector. Scripts like FormMail, when left at their default configuration, could be exploited as open email relays for spam. CERT issued advisory CA-2002-05 in 2002 specifically about FormMail vulnerabilities. Hosting providers began restricting or disabling cgi-bin access, and security-conscious administrators removed the ScriptAlias directive entirely from their Apache configurations.
cgi-bin Today
The cgi-bin directory has not been completely eliminated from the web. It exists in a strange twilight: technically supported, rarely used for new development, but still present in millions of configurations and actively targeted by security scanners.
Apache Still Supports It
Apache HTTP Server still ships with full CGI support through mod_cgi (prefork MPM) and mod_cgid (worker/event MPM). The default httpd.conf on many Linux distributions still includes a commented-out or active ScriptAlias directive pointing to a cgi-bin directory. Apache’s official documentation continues to maintain detailed guides on CGI configuration.
Legacy Systems
Government agencies, universities, banks, and large corporations often have Perl CGI scripts that have been running continuously for 15–25 years. These scripts process internal forms, generate reports, manage inventory systems, and handle data entry. They work. They are understood by the staff who maintain them. Rewriting them in Django or Node.js would cost hundreds of thousands of dollars and introduce new bugs into systems that currently have none. For these organizations, the cgi-bin directory is not legacy — it is production infrastructure.
gitweb.cgi
One of the most prominent CGI scripts still in active use is gitweb, a web-based interface for Git repositories. Written in Perl, gitweb ships with Git itself and provides a browser-based view of commits, diffs, branches, and file trees. It runs as a CGI script (typically at /cgi-bin/gitweb.cgi) and was the default web interface for many Git installations before GitLab, GitHub, and Gitea became dominant. Some self-hosted Git servers still use gitweb today.
Security Scanning
Vulnerability scanners like Nessus, Nikto, and Burp Suite routinely probe /cgi-bin/ paths as part of their standard test suites. They check for known vulnerable scripts: old versions of FormMail, test-cgi, printenv, php.cgi, and dozens of others that have had documented vulnerabilities over the years. The OWASP testing guide includes cgi-bin enumeration as a standard reconnaissance step. The path /cgi-bin/ appearing in server access logs is frequently an indicator of automated vulnerability scanning rather than legitimate traffic.
For server administrators, the recommended practice is clear: if you do not use CGI, remove the ScriptAlias directive from your Apache configuration entirely. An empty cgi-bin directory is not a security risk on its own, but it creates an attack surface that has no benefit.