net.digest February 1998

Click here for RAC sponsor message

Net.digest summarizes helpful technical discussions on the HP 3000 Internet newsgroup. Advice here is offered on a best-effort, Good Samaritan basis. Test these concepts for yourself before applying them to your production or development HP 3000s.

Analysis by John Burke

Does Patch/iX need patching?

Several people posted questions to the newsgroup about missing functionality in Express 3. For example: “I applied Express-3 for MPE/iX 5.5 to our test system a few weeks ago. Yesterday, I attempted to demonstrate the new LISTFILE formats and was surprised to get an error message. What gives?”

HP offered this explanation:
“If you used Patch/iX then the problem is probably that the actual patch which contains the :LISTFILE enhancements was not installed. I believe on PowerPatch 3 this patch is marked as an ‘enhancement’ patch. Enhancement patches will not qualify for installation via Patch/iX automatically. If you want to install enhancement patches with Patch/iX then you must use the ‘force’ feature.”

“We have received plenty of feedback that this interface is not very user friendly and we are looking into the appropriate corrective measures, but in the meantime this is the way it works. For future reference, PowerPatch tapes come with documentation entitled “READ THIS BEFORE INSTALLING.” This issue is supposed to be discussed in that document. We are investigating a more complete solution for the future (including a probable change to the Patch/iX interface to make it easier to use with respect to enhancement patches).”

Fortunately for me, this thread had already appeared on HP3000-L when I went to install Express 3. Had it not, I might well have been caught by this “feature” of Patch/iX, because the “READ THIS BEFORE INSTALLING” text is not terribly illuminating. Nor are some of the patch descriptions.
I do want to add that, for all its problems, Patch/iX is definitely a step in the right direction and you should give it a try.

I wish I could say the same for Express 3 release of MPE/iX. I found it to be a major disappointment. First, it was delayed for many months. Then it took over five weeks to get the tapes after I faxed in my order. And, finally, Express 3 ships with at least two serious IMAGE problems: one will cause a system deadlock (this happened to me twice in 10 days), and the other renders the combination of Third Party Shadowing and Third Party Indexing (such as Bradmark’s Superdex, at our site) inoperable. Both of these were big time showstoppers. To cure these ills there is a patch TIXJXW2A, which if downloaded from the Internet has the not-so-amusing side effect of placing ACDs on all of your IMAGE utilities. If you can, wait for Express 4.

The Tale of the Tape

Maybe I am dating myself, but I still find discussions about things like blocking factors interesting. I hope you do also. Someone asked: “Can anyone tell me if there is a formula to determine the optimum blocking factor if my record length is 156 bytes and I am writing out to tape?”

Noted tape and backup guru Jeff Woods of Unison Software gave an interesting reply which I will quote from liberally:

“Blocking factors for tapes depend on the tape drive and format written. For example, on 9-track drives such as the 7980, the inter-record gap and the length of tape used by the data portion of the block varies depending on tape density (1600 or 6250). On DDS drives it’s more complex still because of something called a super-block which blocks together multiple datablocks into a single media IO. The super-block mechanism makes the user-specified block less important.”

“I believe MPE limits the maximum blocking factor to 255 records per block. Also, the device drivers for the various tape drives have maximum media block sizes of 16K or 32K. To be compatible with the greatest number of drivers, I recommend coming as close to 16K blocks as possible without exceeding 255 records per block. At the same time, I personally find having blocks with wasted bytes following the last record in the block to be somehow untidy. Therefore, I often will actually round the block size down to find one with an integral number of records in a block evenly. This turns out to be relatively straight-forward with a bit of math.

“To use your example of a record size of 156 bytes:
max block size 16K = 16384 bytes
max 156B records in 16K block = 16384/156 = 105.03 = 105 records per block wasted bytes at end of block = 16384-(156*105) = 4 bytes. So a blocking factor of 105 would be pretty good, and is probably optimal.

“However, to meet my personal peeve, let’s see how big a block we can have with no wasted bytes at the end. 156 factors into 2^2*3*13 or 4*39. Looking at a blocksize of 39 sectors:

39*256 = 9984 byte block
blocking factor = 9984/156 = 64 records per block

“This would yield almost as good a use of tape space, depending again on the actual media format used, and each block would contain no unused bytes. I would probably use a BF of 64, but other folks would use 105.”

Make that two ounces of prevention is worth a pound of cure

I had not really thought about it much myself until the question showed up on HP3000-L. After all, how many times do you have to do a reload of an HP 3000? In contrast, it seems to me that we are constantly “rebuilding” our NT servers. Unix? My personal experience has been limited but painful. The HP 3000s? They just keep chugging along. Anyway, the question that started the thread and an edited compilation of responses follows, with contributions from Michael Hensley, Mike Hornsby, Bill Lancaster, and Martha Friedrich:

“I can’t actually remember the last time I had to reload a system, but I am swapping a disk drive on a system and will have to do one. Anyone have a handy dandy check list?”

0) Kick everyone off and shut down networking so no files will fail to STORE because they are in use for writing.
1) As MANAGER.SYS, :RUN BULDACCT;INFO=”@”
2) STORE /;*T;DIRECTORY;SHOW=OFFLINE etc.
3) VSTORE (or however you like to verify your tapes) on a different tape drive if possible.
4) Try to find the time to make a second backup (the second ounce of prevention just in case).
5) Go into SYSGEN and create a new SLT. Again, make two if possible.
When you’re ready to reload:
1) Boot from the SLT
2) Use VOLUTIL to add the rest of your disc drives
3) :RESTORE *T;@.@.SYS;DIRECTORY;CREATE;KEEP;OLDDATE;SHOW=OFFLINE
4) :STREAM BULDJOB1.PUB.SYS (Not necessary provided you have used the DIRECTORY option in both the STORE and RESTORE.) Note also that particularly BULDJOB1 should be secured in some way since it contains every MPE user, group and account password on the system. (If nothing is done and it remains in PUB.SYS it can be read by any user on the system!)
5) :RESTORE *T;/;CREATE;KEEP;OLDDATE;SHOW=OFFLINE
Examine the :RESTORE listings to see if you need to fix anything. [Note: You can re-direct the STORE and RESTORE listings to disc for easier viewing.]

Some further comments:

“Make sure you have all relevant documentation available before starting and have done a dry run of each step.”

“Before making the SLT, I like to re-boot with “GROUP=CONFIG”. This way, I know what group I booted from, I know that I can boot from that group, and I know what group I’m putting on the SLT.”

“My nightly backup jobs run BULDACCT before the backups, so all FULL and PARTIAL backups have current BULDJOBs on them.”

“I like to make sure I have all the latest factory SLT, FOS and SUBSYS tapes handy.”

“My backup job stream purges the BULDJOB files at the end of the job. Since there are no users on the system during backup, the BULDJOB files are safe from snooping.”

“While not necessary, because DIRECTORY is the supported option, BULDACCT has some advantages – particularly if you combine changes in volume sets at the same time you do an INSTALL.”

“A security issue is that step 3 and step 5 may recreate old deleted users. This can create BIG security problems because these users do not have passwords. If you do not specify the CREATE option, necessary files created by former users will not be restored! This is something of a Catch-22.”

“I also make a special store tape of certain accounts (@.@.SYS for example) so I can restore them immediately after the directory is in place.”

“I would suggest another approach. Rather than multiple STOREs, do one STORE with an indirect file. The indirect file would look something like this:

@.@.SYS @.@.ACCT1 @.@.ACCT2 @.@.ACCT3 @.@.@-@.@.SYS-@.@.ACCT1-@.@.ACCT2-@.@.ACCT3

“The STORE command would then look like this:

STORE !indirectfilename;*T;SHOW;DIRECTORY

“This has the advantage of having a single store set while conveniently putting the files which would be restored first at the beginning of the tape set. Note the “-@.@.SYS-.@.ACCT1-@.@.ACCT2-@.@.ACCT3”. This eliminates all the warning messages STORE reports when trying to store files already stored.”

“I would suggest another permutation which has some performance benefits. Since you will be controlling the location of files on the tape, you will also be controlling the location on disk where you put files when you restore them. You can take greater advantage of this if you extend your indirect file scenario to locate the most heavily accessed files early on in the store. This way, when you restore the files, they will be going onto the outer cylinders of the disk, effectively packing the data into more efficiently accessed areas of disk.”




		Click here for RAC sponsor message
		Net.digest summarizes helpful technical discussions on the HP 3000 Internet newsgroup. Advice here is offered on a best-effort, Good Samaritan basis. Test these concepts for yourself before applying them to your production or development HP 3000s. Analysis by John Burke Does Patch/iX need patching? Several people posted questions to the newsgroup about missing functionality in Express 3. For example: “I applied Express-3 for MPE/iX 5.5 to our test system a few weeks ago. Yesterday, I attempted to demonstrate the new LISTFILE formats and was surprised to get an error message. What gives?” HP offered this explanation: “If you used Patch/iX then the problem is probably that the actual patch which contains the :LISTFILE enhancements was not installed. I believe on PowerPatch 3 this patch is marked as an ‘enhancement’ patch. Enhancement patches will not qualify for installation via Patch/iX automatically. If you want to install enhancement patches with Patch/iX then you must use the ‘force’ feature.” “We have received plenty of feedback that this interface is not very user friendly and we are looking into the appropriate corrective measures, but in the meantime this is the way it works. For future reference, PowerPatch tapes come with documentation entitled “READ THIS BEFORE INSTALLING.” This issue is supposed to be discussed in that document. We are investigating a more complete solution for the future (including a probable change to the Patch/iX interface to make it easier to use with respect to enhancement patches).” Fortunately for me, this thread had already appeared on HP3000-L when I went to install Express 3. Had it not, I might well have been caught by this “feature” of Patch/iX, because the “READ THIS BEFORE INSTALLING” text is not terribly illuminating. Nor are some of the patch descriptions. I do want to add that, for all its problems, Patch/iX is definitely a step in the right direction and you should give it a try. I wish I could say the same for Express 3 release of MPE/iX. I found it to be a major disappointment. First, it was delayed for many months. Then it took over five weeks to get the tapes after I faxed in my order. And, finally, Express 3 ships with at least two serious IMAGE problems: one will cause a system deadlock (this happened to me twice in 10 days), and the other renders the combination of Third Party Shadowing and Third Party Indexing (such as Bradmark’s Superdex, at our site) inoperable. Both of these were big time showstoppers. To cure these ills there is a patch TIXJXW2A, which if downloaded from the Internet has the not-so-amusing side effect of placing ACDs on all of your IMAGE utilities. If you can, wait for Express 4. The Tale of the Tape Maybe I am dating myself, but I still find discussions about things like blocking factors interesting. I hope you do also. Someone asked: “Can anyone tell me if there is a formula to determine the optimum blocking factor if my record length is 156 bytes and I am writing out to tape?” Noted tape and backup guru Jeff Woods of Unison Software gave an interesting reply which I will quote from liberally: “Blocking factors for tapes depend on the tape drive and format written. For example, on 9-track drives such as the 7980, the inter-record gap and the length of tape used by the data portion of the block varies depending on tape density (1600 or 6250). On DDS drives it’s more complex still because of something called a super-block which blocks together multiple datablocks into a single media IO. The super-block mechanism makes the user-specified block less important.” “I believe MPE limits the maximum blocking factor to 255 records per block. Also, the device drivers for the various tape drives have maximum media block sizes of 16K or 32K. To be compatible with the greatest number of drivers, I recommend coming as close to 16K blocks as possible without exceeding 255 records per block. At the same time, I personally find having blocks with wasted bytes following the last record in the block to be somehow untidy. Therefore, I often will actually round the block size down to find one with an integral number of records in a block evenly. This turns out to be relatively straight-forward with a bit of math. “To use your example of a record size of 156 bytes: max block size 16K = 16384 bytes max 156B records in 16K block = 16384/156 = 105.03 = 105 records per block wasted bytes at end of block = 16384-(156105) = 4 bytes. So a blocking factor of 105 would be pretty good, and is probably optimal. “However, to meet my personal peeve, let’s see how big a block we can have with no wasted bytes at the end. 156 factors into 2^2313 or 439. Looking at a blocksize of 39 sectors: 39256 = 9984 byte block blocking factor = 9984/156 = 64 records per block “This would yield almost as good a use of tape space, depending again on the actual media format used, and each block would contain no unused bytes. I would probably use a BF of 64, but other folks would use 105.” Make that two ounces of prevention is worth a pound of cure* I had not really thought about it much myself until the question showed up on HP3000-L. After all, how many times do you have to do a reload of an HP 3000? In contrast, it seems to me that we are constantly “rebuilding” our NT servers. Unix? My personal experience has been limited but painful. The HP 3000s? They just keep chugging along. Anyway, the question that started the thread and an edited compilation of responses follows, with contributions from Michael Hensley, Mike Hornsby, Bill Lancaster, and Martha Friedrich: “I can’t actually remember the last time I had to reload a system, but I am swapping a disk drive on a system and will have to do one. Anyone have a handy dandy check list?” 0) Kick everyone off and shut down networking so no files will fail to STORE because they are in use for writing. 1) As MANAGER.SYS, `:RUN BULDACCT;INFO=”@”` 2) `STORE /;T;DIRECTORY;SHOW=OFFLINE` etc. 3) `VSTORE` (or however you like to verify your tapes) on a different tape drive if possible. 4) Try to find the time to make a second backup (the second ounce of prevention just in case). 5) Go into SYSGEN and create a new SLT. Again, make two if possible. When you’re ready to reload: 1) Boot from the SLT 2) Use VOLUTIL to add the rest of your disc drives 3) `:RESTORE T;@.@.SYS;DIRECTORY;CREATE;KEEP;OLDDATE;SHOW=OFFLINE` 4) `:STREAM BULDJOB1.PUB.SYS` (Not necessary provided you have used the DIRECTORY option in both the STORE and RESTORE.) Note also that particularly BULDJOB1 should be secured in some way since it contains every MPE user, group and account password on the system. (If nothing is done and it remains in PUB.SYS it can be read by any user on the system!) 5) `:RESTORE T;/;CREATE;KEEP;OLDDATE;SHOW=OFFLINE` Examine the :RESTORE listings to see if you need to fix anything. [Note: You can re-direct the STORE and RESTORE listings to disc for easier viewing.] Some further comments: “Make sure you have all relevant documentation available before starting and have done a dry run of each step.” “Before making the SLT, I like to re-boot with “`GROUP=CONFIG`”. This way, I know what group I booted from, I know that I can boot from that group, and I know what group I’m putting on the SLT.” “My nightly backup jobs run BULDACCT before the backups, so all FULL and PARTIAL backups have current BULDJOBs on them.” “I like to make sure I have all the latest factory SLT, FOS and SUBSYS tapes handy.” “My backup job stream purges the BULDJOB files at the end of the job. Since there are no users on the system during backup, the BULDJOB files are safe from snooping.” “While not necessary, because DIRECTORY is the supported option, BULDACCT has some advantages – particularly if you combine changes in volume sets at the same time you do an INSTALL.” “A security issue is that step 3 and step 5 may recreate old deleted users. This can create BIG security problems because these users do not have passwords. If you do not specify the CREATE option, necessary files created by former users will not be restored! This is something of a Catch-22.” “I also make a special store tape of certain accounts (@.@.SYS for example) so I can restore them immediately after the directory is in place.” “I would suggest another approach. Rather than multiple STOREs, do one STORE with an indirect file. The indirect file would look something like this: `@.@.SYS @.@.ACCT1 @.@.ACCT2 @.@.ACCT3 @.@.@-@.@.SYS-@.@.ACCT1-@.@.ACCT2-@.@.ACCT3` “The STORE command would then look like this: `STORE !indirectfilename;T;SHOW;DIRECTORY` “This has the advantage of having a single store set while conveniently putting the files which would be restored first at the beginning of the tape set. Note the “`-@.@.SYS-.@.ACCT1-@.@.ACCT2-@.@.ACCT3`”. This eliminates all the warning messages STORE reports when trying to store files already stored.” “I would suggest another permutation which has some performance benefits. Since you will be controlling the location of files on the tape, you will also be controlling the location on disk where you put files when you restore them. You can take greater advantage of this if you extend your indirect file scenario to locate the most heavily accessed files early on in the store. This way, when you restore the files, they will be going onto the outer cylinders of the disk, effectively packing the data into more efficiently accessed areas of disk.”