ftpbot1.pl A bot to help FTP searches	Back to bots.htm
March 2000	by DarkWyrm
	Courtesy of Fravia's searchlores.org
fra_00xx 980921 darkwyrm 1000 BO PC	DarkWyrm will -I hope- develop this project further and send more info and more projects. The help of all the readers (that's you, duh) that will want to comment or contribute is welcome. Feedback: darkwyrm(at)home(point)com
	There is a crack, a crack in everything That's how the light gets in
Rating	(X)Beginner ( )Intermediate ( )Advanced ( )Expert

Introduction

Sifting through the indexing done by archie and FTP search sometimes *still* doesn't prove to be enough to get a file from a site. This is a basic tool I am currently developing which helps for when you think a file is on a site, but don't want to slog through the tedium of finding it yourself.

Tools required

Perl (standard on Linux and readily available for other platforms)
WWW access
text editor (I prefer UltraEdit)

Essay

FTP Search, while useful in its own right like any other search engine, still suffers the same limitations any other search engine has - a, well, obscene amount of information to index and constant change of this information from moment to moment. To work around this, you need to use a tool that is not nearly as broad in scope as a search engine. A bot will work nicely.

I won't explain the code too much -- if you don't know Perl, learn it. If you do, then my first programming project will likely be an exercise in boredom. The code is fairly well-commented and straight-forward. If you wish to find what you want, you must have a desire to be good at looking for it.

In short, this script takes two arguments - a URL and a filename. It logs in and literally reads every directory and filename on the site. If it finds another directory, it adds it to the list of paths to search. If not, it checks to see if the file is a match. Only the first result is displayed on the screen. No wildcards, no cool file I/O, and almost no error checking for now. ;^) This was tested on Win32 Perl, but shouldn't cause any problem on other platforms. As a bot, it doesn't play nice - it wastes no time in searching for stuff, but I doubt it will cause any serious bandwidth problems. Don't blame me if some nice admin sends you a very, um, polite e-mail about this.

  #! /usr/bin/perl -w 
 ############################################################################
  ###########
  # FTP Site Searchbot
  ############################################################################
  ########### 
 # Current functionality - searches site & exits if it finds a match. Tweak
  to your
  # heart's content 
 # Mods to include
  use Net::FTP;
  use Cwd;
  use Getopt::Std; 
 $ftp =""; 
 $remotehost = '';
  $remotedir = '/';   # current path on remote host
  $username = 'anonymous';
  $password = '[email protected]';
  $isdir=0;
  $pathindex = 0;
  $currentfile = "";   # placeholder var for file name in current entry
  $filename = "";     # Name of file to be searched for
  $done=0;      # Exit flag
  @dirlisting = "";    # Holds the listing generated by a dir command
  @currententry="";    # Current entry in question in a dirlisting
  @pathlist="/";     # List of all the directories on a site 
 # Start off the process by getting host and file from cmdline
  $remotehost=lc($ARGV[0]);
  $filename=lc($ARGV[1]); 
 # Did they even give us a host and filename?
  if ( !($remotehost) or !($filename) )
  { print "\nSyntax:\n\tftpbot.pl URL filename.xxx\n";
   exit 4;
  } 
 # Log in
  if ($ftp = Net::FTP->new("$remotehost",0))
   { print "Connected to host $remotehost.\n"; }
  else
  { print "Couldn't connect to host $remotehost.\n";
   exit 5;
  } 
 print "Logging in as $username.\n"; 
 if ($ftp->login("$username","$password"))
  { print "Login successful.\n";
  }
  else
  { print "Couldn't login";
   exit 10;
  } 
 
  # Start from root and search 
 while(defined($pathlist[$pathindex]))
  { 
  # Change to new directory
   $remotedir = $pathlist[$pathindex];
   $ftp->cwd("$remotedir"); 
  # Get new directory listing
   @dirlisting = $ftp->dir(); 
  #extract directories from listing, save, and print
   for($i=0; $i<=$#dirlisting; $i++)
   {
    @currententry = split / /, $dirlisting[$i]; 
   # Skip any 'total files' listings
    if( (lc $currententry[0]) eq 'total') {next;}
    $isdir = substr($currententry[0],0,1); 
   $currentfile = lc($currententry[$#currententry]); 
 
    # If entry is a directory, save and print
    if($isdir eq 'd') 
 
     # Construct full path and place at end of list
     push @pathlist, ($remotedir . $currentfile . '/');
    }
    elsif($currentfile eq $filename)
    { # This sucker's a match. Print & quit.
     print "$remotedir" . "$currentfile" . '/';
     $done = 1;
     last;
    }
   }
   $pathindex++;
   if ($done) {last;}
  } 
 print "\nClosing connection to $remotehost.\n";
  $ftp->quit;
  

Final Notes

This is still a work in progress. Plenty of bots are out there, but if you want something done right, do it your own damn self. You know what your own search needs and wants are, take what I've started and make it something a bit more complex and much more useful to yourself. I certainly plan on it. I might even release a later version if there's interest... Being a leech can only get you so far - I should know. Building on one anothers' strengths can we stand against the commercial zombie masters...

fravia's searchlores.org

Back to bots.htm