By Wirt Atmar

We have been using one of our HP 3000 Series 918s to shadow our primary development 918, using the relatively new MPE-to-MPE FTP capabilities of MPE. The purpose of doing this wasn’t to eliminate tape backups, but rather to insure that if we lost one machine — and all of the recent backup tapes that tend to lie around it — we’d have another duplicate machine, completely ready to run at a moment’s notice in a completely separate building, far enough away where the possibility of both being destroyed by fire was highly unlikely.

Firstly, the job is extremely simple — and of course free. Indeed, there’s nothing complicated about any of it, nor are there any costs other than a few minutes’ time. The job is constructed as shown below, streaming itself to run at 3 AM every morning:

!job ftpxfer,user1.acct1,group1
!ftp 192.168.1.1
user user1.acct1,group1 psw,psw
prompt
mget *
quit
!set stdlist=delete
!eoj
!job ftpxfer,user2.acct2,group2
!ftp 192.168.1.1
user user2.acct2,group2 psw,psw
prompt
mget *
quit
!set stdlist=delete
!eoj
!job ftpxfer,user3.acct3,group3
!ftp 192.168.1.1
user user3.acct3,group3 psw,psw
prompt
mget *
quit
!set stdlist=delete
!eoj
!job ftpxfer,user4.acct4,group4
!ftp 192.168.1.1
user user4.acct4,group4 psw,psw
prompt
mget *
quit
!set stdlist=delete
!eoj
.
.
.
!job ftpxfer,manager.sys,manager
!pause 90
!stream ftpjob.manager.sys;at=03:00
!set stdlist=delete
!eoj

The primary development machine is running MPE/iX 5.5 PowerPatch 7, and the shadower running MPE/iX 5.5 PowerPatch 4. The primary machine has only 4Gb of disk; the shadower has 8Gb. There is a 10Mbit LAN between the two machines.

The backup of the primary machine onto a DDS-2 drive takes about 45 minutes to make a full CSLT/store copy of all files (approximately. 3.5Gb), using hardware compression, but there’s no need for us to transfer all of the user files from one machine to the other on a regular basis. The seven jobs represented in the jobfile above transfer only the regularly active development and corporate accounts and their databases. These seven jobs represent about 2.5Gb worth of material.

My initial experiments indicated that transfers across the LAN were just about as fast as DDS/compression on backups. That still seems to be true. A complete CSLT/store backup of the primary takes about 45 minutes; the transfer of about half that material across the LAN takes 23 minutes.

There is a caveat in that 23-minute number, however. That transfer rate occurs when we allow all seven FTP jobs to run in parallel. A 10Mbit LAN is the equivalent of six T1 lines. If the seven jobs all run in parallel, we seem to consume about 60 percent of the LAN’s bandwidth — leaving the equivalent of two T1s available for all other internal traffic, and that seems more than enough. While this intense internal traffic is flowing, normal terminal communications or Web page draws seem unaffected.

However, with all seven jobs running simultaneously, the disks on both machines are being exercised to their maximums, to the point that I consider it excessive wear on the disks. They’re really clattering. Thus, we now set the job limit to just one above the background jobs, so that the seven jobs execute in single file; however, doing this increases the transfer time from 23 minutes to 64 minutes.

A good portion of a single-file’s FTP bandwidth (dead-time) is consumed in just the negotiation between the two machines. Once the file transfer begins flowing, the data moves at a good speed. When all seven jobs are running simultaneously, whatever dead time on the LAN is left over by one job is filled in by one or several of the others. Nevertheless, the excess wear on the disks doesn’t seem worth the additional 40 minutes you save, especially at three o’clock in the morning.

I’ve tried multiple FTP syntaxes, but the one used in the job above seems by far to be the most certain. However, it only transfers MPE-named files, but that’s all we currently use, so it’s not a problem for us. IMAGE databases, KSAM files, and all “regular” files transfer with no problem. Symbolic links transfer, but seem to lose their “symbolicness” in the process. HFS files don’t transfer at all, using this syntax and the MPE version we’re currently on.

I’m also planning on upgrading both machines to 6.5. Once that’s done, we’ll be able to use Jeff Vance’s new Store-to-Disk option. If my understanding is correct, the great advantage of doing this is that we’ll be able to get all of the advantages of using the STORE command and assemble the store material into one file that can be FTPed between the two machines, and then RESTORED on the shadow.

However, I’ve never been particularly fond of partial backups, but in order to make the STD option work well, that’s what we’ll have to do. The FTP jobs above transfer every file in the specified groups and accounts, regardless of the recency of their modifications.

One final note: FTPing 2.5Gb of files between the two machines does not have the same impact on users as does a STORE-like backup does, where all of the files are marked for some time for exclusive access. Only one file at a time is locked as the MGET process walks through the file list. That attribute can obviously be either good or bad, depending on individual circumstances. Nonetheless, using the simple job that I’ve outlined above, that is the current behavior.

Overall, I’m quite tickled as to how it’s working. If we should lose the primary machine, we are (nearly) guaranteed of having a perfect replicate machine, available and ready to go, in building three.

Wirt Atmar is founder of AICS Research, makers of the QueryCalc application for HP 3000s and the QCTerm terminal emulator.