DOS FTP Automation

COMMAND-LINE FTP AUTOMATION

How to use the native FTP program that is integrated into DOS for the automation of transfers to or from any ftp host.

AUTHOR

    Bryant Martin,
    mail-production.com/

OVERVIEW

How to use the built-in ftp program provided within MS Windows systems is an often-asked question. Automated FTP sessions are possible if the reader understands the nature of the command-line FTP program and can write a batch program to call this built-in program, and provide a login script that can supply the information and parameters that command-line FTP needs in order to operate in an automated, preset fashion.

Command-line FTP can be powerful, but it is arcane and basic. It uses a standard port-mode connection and does not support passive mode (PASV) connections.

For these reasons, most any ftp host should be able to negotiate the basic functions described here for that host to have validity as a normal or common FTP host system.

The methods shown here, when executed correctly as shown, will work for any normally functioning ftp host; and they will work remotely from any normal MS Windows client wishing to employ them.

This means that the purchase of FTP client-side software should not needed to conduct very good FTP sessions from MS Windows computers. Those programs are excellent to use and can help users organize and store FTP parameters for a large number of hosts, but, as we will next see, they are not necessary.

DEFINITIONS

    DOS - disk operating system
    FTP - file transfer protocol
    PORT - access path for network-addressed transfer packets  
    PASV FTP - client connects to server for data transfer (passive)
    PORT FTP - server connects to client for data transfer (active)
    TCP - transmission control protocol
    FTP CLIENT - remote workstation requesting FTP from server
    FTP SERVER - all FTP communication is by TCP with no UDP function
      main port (usually 21 control) for command (USER, PASS, CWD ...) 
      data port (main port - 1) for data transfer (directory listing,
      upload & download)
    UDP - user datagram protocol - not used in FTP
    WINDOWS® - ©Microsoft Corporation

DISCUSSION

Some FTP high-level software is so badly developed that it is often a good idea to drop back to the basics to clear some of the problems encountered when misconfigured high-level programs fail to operate as expected while talking to a host server. These problems are normally found because of confusion about who's running the show, client or host, and whether connections shall be by passive-mode or by port-mode FTP. See our separate paper discussing passive- and active-mode FTP later for a developed discussion of this topic.

We find today there are many clients and servers that are running firewalls that are controlling the port access to the raw ports or listening daemons. When blockages result it is nearly impossible for active-mode FTP sessions to operate until all permissions and calls for the expected use-port ranges are programmed, cleared and tested. We will avoid this confusion by using FTP's default control and data ports in a port-mode ftp connection session.

We will be using FTP's default tcp port 21 for a control connection, and transfer data using the port requested by the client in an active-mode session. Example: if the client requests connection PORT 192,168,0,10,5,114 from the server, the server must connect to the client for a data transfer link at 192.168.0.10 on port 5*256+114 = 1394.

    Client > PORT 192,168,0,10,5,114 
    Server > 200 PORT command successful.

DIAGNOSTICS

For diagnostics, we have a handful of matters to confirm:

    1. client has command-mode FTP available.
    2. client has an active network to the internet.
    3. client has permission for use tcp port 21.
    4. client has used a user name the server has authorized.
    5. client has supplied correct password for this user.
    6. client has dns service for the domain name used.
    7. client has correctly written batch program.
    8. client has correctly written login script.
    9. client has used valid ftp commands to control the session.
   10. client can receive data through firewall at port requested
   11. server has permission to use tcp ports 21 and 20.
   12. server has permission to use client's requested data port.
   13. server is running an ftpd, properly configured. (a daemon)
   14. server is expecting connect from client requesting connection.
   15. server is online.

If the reader can confirm all the above, then a fix of any failure can be narrowed down to the list above. Most failures will occur on point 10 above, when the client's firewall is blocking the return of the server's data link.

Example: how do I know as a client if I have command-line ftp available?

Here's a way to check:

Open a DOS prompt window. Search for ``ms-dos'' using your search or find features (right-click ``start''). Locate MS-DOS Prompt shortcut and do a right-click and choose ``create shortcut'' place a copy of this shortcut on the local desktop.

Use the shortcut just located to open a DOS window. This will open a new command interpreter, called a DOS prompt, or operator console, for system commands to be entered at the computer's operator console, the way all things were done on computers before Windows gave us another interface called "user-friendly."

To exit the DOS environment and go back to Windows, the command DOS expects is "exit" or "EXIT". That will close any windowed or full-screen display and send you back to Windows.

The combination command, ALT+ENTER, will switch your DOS command prompt from a full screen to a windowed view, or vice versa. The windowed view is fine for our purposes here and is the default DOS-console view, or prompt,

    from the command prompt enter command

    ==>ftp

    if the ftp prompt appears, then program ftp is now running

    to exit this program, enter the command,
    
    bye

    you will then be returned to the DOS system prompt

HINT

When I first began to use an IBM mainframe computer in earnest as a programmer, an old hand told me this,

There are many ways of doing most anything on a computer, not just one way. Sometimes I do it this way, sometimes I do it that way, I sometimes I do it the other way. All of them work, and are fine. Some are better in certain situations, or better for a particular person's understanding at this time.

This is good to remember because you will say to me or another expert, "That's not the way Sam does it!" Right, Sam probably does it the other way; but if you ask Sam about this way he'll say, ``Yes, this way is fine, I just told you the other way because I thought you'd understand it better, and because it's the way our lessons so far were being developed.''

A computer is like a house with many doors. If your system of learning has thus far said that, to begin, we always walk around back, knock twice and come in through the kitchen door, that's fine. But later you will discover in another study that you can also get in by going to the front door, ringing the doorbell once, and coming on in. You may not then be in the kitchen, as expected, but you're in the house just as well; and from that point you'll have to figure out where you're going inside this very large mansion.

An experienced computer user knows there are side doors, underground tunnels, as well as unlocked bathroom windows.

WRITING A BATCH PROGRAM

A batch program, or batch file, is an executable file with a filename extension of BAT, or dot bat.

All it really is is a stack of some commands that one might enter separately, one at a time, from the command console.

But the above generalization truly depends on what your definition of ``is'' is, as there are some major differences in that you can do things in a batch program you cannot do at a prompt because programs have support for a language protocol, which a mere prompt does not.

It's equivalent to a control-language program, or CL program used on other computers. All this implies is that, yes, this file, or program, will contain raw commands that are legal commands commonly entered at the operator console.

The CL method of storing command in a certain order, and running them at a certain time is a convenient way of automating a routine computer operation so that, gee, I can store all this hard-to-write stuff in this one neat little file, all tested and debugged, and run it any time I like; and hey, it works the same as if I'd actually been there and entered the commands one at a time without making a single keying error on the console. It also spares me a long drive from home to office if I decide to run this thing from a timed-scheduled release, like Window's Task Scheduler.

One really begins to see the power of this automation when his or her imagination runs away in knowing that one CL program can call another, and all things being error free (and they can also confirm many levels of error) that CL can call another, and so on forever.

Any errors are cured by special recovery programs called when error occurs. These are known as `handlers,'' for they handle errors that are trapped by type, and they take careful experience and debugging to run, again, error-free. The handlers are commonly just sub-routines held within a larger program or batch file, and they require knowledge of how to write sub-procedures as a programmer becomes more experienced.

I once knew a radio station manager who went to England for two weeks, without the university-owned station's knowledge, and everything went off without a hitch. I would listen in amusement, "Hello, everyone. It's Tuesday morning, 7:34, and that was Brahms Symphony No.2, in D Major, performed by thus and so orchestra...''.

I was amazed and almost started to call him there at work to exclaim, ``Peter, it's working wonderfully; you've got 'em bamboozled...'' until I remembered he was away in another continent, and quite confident that all was well back in Richmond, Virginia, 3000 miles away. All it took was for the engineer and cohort for the masquerade to start a dated ``Tuesday'' cassette at precisely the right moment, and start all the gigantic tape libraries from which all his predetermined settings called hundreds of precisely timed and cued musical recordings, all with no human intervention, the same way he had broadcast all the music, while there, for years.

This graduate student's first several trips abroad were probably with the station's well wishes; beyond those his itinerary became a bit tenuous. But there is always a nice salvation to be found in a carefully planned and executed automation of life's daily routines.

CODING OF BATCH FILE

For this automation we'll need one simple batch program named FTP1.BAT that contains this,

    @echo off
    ftp -s:ftplogin.txt %1

    or, optionally, it may have the host server's name
    hard-coded into the file in this manner,

    @echo off
    ftp -s:ftplogin.txt ftp.cms-mpc.com

    or this,

    @echo off
    ftp -s:ftplogin.txt 65.246.15.2

In the first instance, above, the %1 is a convenient way of having our batch program pick up the meaning of %1 from the command prompt's first parameter following the command name, as in,

    ==>ftp1 ftp.cms-mpc.com

In this case, the ``ftp.cms-mpc.com'' is brought into the batch file as literally ``ftp.cms-mpc.com'' for every instance of the holder "%1" found in that file.

Note: any completed batch file then becomes known to the resident system as a valid ``command'' because a batch file is ``executable,'' meaning, simply, it will ``run'' or execute.

One must make certain of this command's location is within the systems known ``paths,'' which are the directories where the system searches for executable files to run before it just gives up and reports back to the console,

    ftp1 is not a recognizable internal or external command,
    operable program, or batch file

You can avoid program-location errors by changing directories to the directory where file FTP1.BAT resides, because the operating system will look first in this current directory for commands or programs before it looks elsewhere in the path variable's specified locations .

To see your path variable, just call this command,

    ==>set path
    or
    ==>set | more

We won't go into a discussion of paths here, or how to set them. But, basically, the path is where your system will look for a program to run after you have called that program by name from the operator console, or when any running program has called for it. You can name your batch file, once decided upon, any valid name. Just don't name it FTP.BAT because your system could confuse it with the resident program ``ftp.exe'' and cause you some miffed experiences until you discovered this error.

You can use program notepad.exe or program edit.exe to write and maintain your batch program and your script text file.

The syntax to call them is:

    edit ftp1.bat
        or 
    start notepad ftp1.bat

SWITCHING TO A DISK DIRECTORY

First we must make sure we're logged into the correct disk drive, and to do this we issue DOS command,

    ==>c:

This assures we are logged to the disk drive c:. From here we will change directories to the one we want on drive c:

    ==>cd \trf

Assuming that c:\trf exists, this is where we now are for our current or ``default'' disk directory. To see where we are, just enter command,

     dir | more

Note: the "|" character is above the "\" key, and this is the ``pipe'' character, which means to take the screen output from "dir" and pipe it through "more" so we can see it one screen at a time, if it's more than one screenful.

Another way is to set your command prompt with a nice telling instruction,

    ==>prompt $_$d $b $t$h$h$h$_$p $q$q$g

    (there are some Q's an G's in there)

After this is set you'll know the current directory, as well as the date and time. Just enter ``prompt $p$g'' if in a hurry. There are ways to start this in the autoexec for your system so it runs when the system starts.

Search for ``autoexec'' and insert this setting it into the file that has any other ``set'' commands; or you may search for help information on setting the environmental variables for your particular operating system.

These control files are normally named "autoexec.bat" or "autoexec.nt", and may need to be created if they are absent from the system. In some cases the Windows registry has to changed to observe (vary to state `1' for true) these startup files.

WRITING A LOGIN SCRIPT

Okay, you may be saying, this is a bit much. If DOS batch files are so great, why do I need a login script?

Good question. Normally they are very smart, and all that you'll need to run your computer.

Here are some tidbits, though: we must think of program control.

In a batch file you may one day decide to call an executable program or another batch program and be miffed to discover that after you call it and it runs, all else fails and nothing else in your long batch program runs at all.

What happened? What has happened is that your batch program gave up control of the system when is called your exe program. It said,

    start mygreatprogram
    
    and then it said to DOS

    bye, bye. I'm through now and I'm outta here...

    program mygreatprogram has the floor now, and
    you'll will have to go wherever he sends you when he's done

This can be easily cured by using the explicit ``call'' command.

``call'' says to DOS: okay, run this program, but wait: when you're done, no matter what, come back here to me. I've got fifteen more instructions and calls for you to run after this.

It also lets DOS know that batch will be staying open, right at this point, and DOS can resume running its instructions from the next line, and not from the head of the batch file, as those lines are already quite done.

This is to say that from a batch program, if you name a program's name on a line, DOS will go find him and run him. But if we don't say

    ==>call somegreatprogram

your batch file can lose control of the operating system and close without any warning to you, the operator. This will always happen if you call another batch program.

This is the issue we have with FTP1.BAT. He's calling another program named ftp.exe, and ftp.exe has the floor after that.

In fact FTP.EXE is taking the host name from FTP1.BAT and using it to call host ftp.cms-mpc.com on tcp port 21.

Beyond that, anything that host ftp.cms-mpc.com asks for, such as a user name, will be unknown to little program FTP.EXE. He just cannot answer big boss host ftp.cms-mpc.com at all.

This is my why our batch program cleverly has encoded into itself the script notation switch, -s

It says to DOS,

    hey brother
 
    I'm going to call a program named ftp.exe
    and I'm going to call him with a switch on

Program FTP.EXE has been written with the knowledge that he may someday be called with a switch on, namely switch "-s" which is the script switch.

This says to program FTP.EXE, okay when you've connected to host ftp.cms-mpc.com on port 21, and that host says, ``what's your username'', you will not go to console and flash on the screen

    User (cms-mpc.com: (none)):

Instead you will go a script file named ftplogin.txt and look in its line one and get that name and tell that host, yes, I'm

    username

That having been sent to host ftp.cms-mpc.com, the host will want to know this user's password.

At that point ftp.exe has to know you're not at the console and can't reply, so get the password and anything else we'll need in this session from our friend, ftplogin.txt

ftplogin.txt has all the answers, and he's got all the commands that we have for host ftp.cms-mpc.com today.

You will note that FTP1.BAT is quite impotent say anything at this point.

He started ftp.exe with a switch, and then he said to DOS, ``let those two have it from here. I'm standing by. ftp.exe and his pal ftplogin.txt can handle all else from here.''

FTP1.BAT is now quite impotent and just standing by. In fact, had he passed control to another batch program, he'd be closed. He will remain standing by if an executable program is called, such as ftp.exe, but he can't do a thing until ftp.exe hands off system control back to this calling batch program, or back to DOS, if that batch file is closed or has nothing further to do.

Had FTP1.BAT called ftp.exe saying,

    call ftp -s:ftplogin.txt ftp.cms-mpc.com

he'd still be standing by and ready to resume control later. But he need not be open at all, since script ftplogin.txt has all the details that are needed for the completion of the user's complete session.

It is also not likely for DOS to go back and forth from ftp.exe and FTP1.BAT and share system control. Only controlling program ftp.exe could ask DOS to do that, and he has no knowledge of your batch file FTP1.BAT because you just named and wrote him today. ftp.exe has been around for a while, as he dates back to the date of the file copied from your Windows installation disks, and he's an old timer.

Once DOS passes control to ftp.exe, he has control all by himself, with the reservation that he will need some help from ftplogin.txt, as discussed above.

Thus, with all this now clearly understood, we'll see that ftplogin.txt has all the login information, and all the session control commands that this session needs:

The ``-s'' switch's presence instructs program ftp.exe: ask my associate for everything requested from this host, and for all FTP commands, and

    when the user name is wanted

        send his line one

    when a password is wanted

        send his line two

    when host wants to know what to do

        send his line three

    when line three's done

        send his line four

    when we've done all we want to do at this host,
    tell him we're done now and we're leaving by sending
    his very last line

        bye

This tells host that user has done all he wants, and host may drop our connection now, telling user,

    bye bye

At this finishing point DOS will regain system control and program ftp.exe will be quite finished; and, there are no programs in control from this command prompt, so the prompt now awaits a new command.

DOS is now awaiting what next to do, and will wait there until the power fails or the system crashes.

But to get back to Windows, you can simply say,

    exit

at the command prompt and the command interpreter will close and Windows will resume active system control, and this interpreter will be quite done and also closed down and be no longer in the memory of this system.

If you like you can put these additional commands into FTP1.BAT, after he calls ftp.exe,

    pause (wait for user to read transfer details)

    cls (clear the console screen)
        
    exit (close interpreter and go whence you came)

CONTENTS OF LOGIN SCRIPT

File ftplogin.txt must include

    username
    password
    cd incoming  (may not be needed on other ftp servers) 
    bin
    put uploadfile.zip
    bye

You will note that text file ftploging.txt basically has two major divisions:

Part one is the actual login stuff and occupies line one and two only.

Part two is all the commands that are needed in this session:

    cd incoming

        switch to server directory /incoming/ if needed

    bin

        tell host we'll be using binary transfer protocol
        in this session, mode I

    put uploadfile.zip
    
        tells host we're sending file uploadfile.zip using
        tcp port 21

    bye

        tells host we have nothing further to do and tell
        him to drop our connection after telling us goodbye

FTP is wonderful because, unlike other connection ports, such as port 80 used by web browsers, it maintains a constant connection with the host at all times, until the hosts waits, say, 900 seconds and drops a client connection after a preset timeout period that is decided by that host administrator.

This is why if you leave a client connected to a host for a long time and ask nothing of the host, the host will drop your connection after a preset time to conserve its resources.

Port 80 browser connections drop immediately after the page server sends your page requests. They await another port 80 call from your browser, such as a mouse click, or refresh order, to renew sharing system resources with you. This is why they can have so many simultaneous users on line. They only reserve connection resources with them when it's absolutely necessary.

This is also why port 80 browsers make poor ftp clients. They are trained to act as web browsers and can at times drop a ready status from a host.

IN A HURRY

Make two files:

/1/ FTP1.BAT

    @echo off
    ftp -s:ftplogin.txt ftp.cms-mpc.com

/2/ FTPLOGIN.TXT

    username
    password
    cd incoming
    bin
    put uploadfile.zip
    bye

Try to run the session by running FTP1.BAT.

If all works, skip reading all the above background and discussion, consider yourself advanced, and move on.

I want to run dos ftp interactively...using built-in ftp manually

How To Transfer Multiple Files Using FTP in Batch Mode

Your ftplogin script file would look something like this. Remember, the script file is called by the FTP program, as discussed above, by using the FTP program's script switch [-s:scriptfilename].

    username        ;send username 
    password        :send password
    cd /backup/archives/trf     ;change to desired remote directory 
    bin         ;put in binary mode 
    prompt      ;toggle confirmation prompting to off
    mput c:\trf\*.zip       ;send all files with the .zip extension
    bye         ;quit the FTP program

OTHER HELPFUL FTP COMMANDS

Some ftp clients support other important FTP commands.

Binary mode is essential for transferring compressed archives, such as ZIP files, or for transferring executable .exe programs, and for these binary types the binary (MODE I) protocol must be used.

Note that the "ASCII" option will not transfer files other than ASCII text successfully.

Files transferred in ASCII mode, when binary mode is needed, will be unusable on the user's end.

TROUBLESHOOTING

Remove client firewall or switch it momentarily to DMZ host mode. Begin a manual command-line session to host, and turn ``debug'' on and record session screen findings to learn where a problem may be occurring.

OTHER FTP COMMANDS

Many commands may be abbreviated, eg,: bin for binary. Here are some commands:

    !           delete      mdelete     proxy       runique
    $           debug       mdir        sendport    send
    account     dir         mget        put         size
    append      disconnect  mkdir       pwd         status
    ascii       form        mls         quit        struct
    bell        get         mode        quote       sunique
    binary      glob        modtime     recv        system
    bye         hash        mput        remotehelp  tenex
    case        help        nmap        rstatus     trace
    cd          image       nlist       rhelp       type
    cdup        lcd         ntrans      rename      user
    close       ls          open        reset       verbose
    cr          macdef      prompt      rmdir       ?

LIST OF FTP RETURN CODES

This list includes most of the FTP codes in RFC 959. Some FTP servers may issue replies that are slightly different.

Copyright  © 2003 mail-production.com^® created 2000 -- last modified 2003-05-12