Robelle Tech: Important Suprtool Performance Notes

Robelle Sponsor Message

February 2004

Get the most out of your Robelle Tools

Important Suprtool Performance Notes

By Neil Armstrong

Suprtool extracts data very quickly; in fact, at times, HP asked me how we can throttle it back. Suprtool tends to push the envelope on disk IO, as it completes processing of each block it reads so quickly that at times, depending on the system configuration, some resources get pushed to the limit. This resource utilization became more of an issue with MPE/iX 6.5, due to changes in the memory manager and is more prevalent in systems with large amounts of memory. Symptoms vary depending on whether you are working with an N-Class system or a non-N-Class.

Happily, HP was most helpful in providing methods to help tune Suprtool for each system. First, some history on key developments that occurred for MPE/iX 6.5, 7.0 and N-class servers.

During the development of MPE/iX 6.5, HP contacted us to come down and get Suprtool to work with Large Files and MPE/iX 6.5. It was a requirement that Suprtool be able to handle Large Files and MPE/iX 6.5 This proved to be a challenge, not because the work was particularly technical, but rather because on the first machine we used the network would not stay up long enough for us to work on anything other than the console. The first trip was dedicated to looking at what would fail, detecting MPE/iX 6.5 and turning off certain options until HP had time to get more issues fixed.

The second trip was dedicated to turning some of those options back on, fixing Sort to work with Large Files and getting Suprlink and STExport to work. After the second trip, we were pretty much ready for Suprtool to be released with the support for large file features and compliant with MPE/iX 6.5.

However, as MPE/iX 6.5 rolled out, we began to get reports of performance issues, which were linked back to some “near release” changes to the memory manager. With help from some great lab people, such as Kevin Cooper and Bill Cadier, we quickly determined the source of the problem as well as implemented a solution on both sides.

The Problem

HP identified the performance problem to be related to our prefetching of pages of data so that subsequent reads are completed quickly. Simply put, we make data available in memory ahead of where Suprtool disc reads are done, in such a way that IOs are reduced. Because of a memory management change, these pages were being kept in memory longer than previously, which forced the memory manager to read down a longer chain of pages and increased the amount of work the system had to do. At times, the problem presented itself as a high number of disk IOs waiting to complete while the system was doing so much work in-between disk IOs. As a temporary work around, we had customers turn off prefetching with a command of Set Prefetch 0.

The Solution

The solution was to reduce the number of pages that the system had to chain through, by releasing them from memory earlier. HP made a enhancement to one of their low-level system calls, which allowed us to invoke an option for releasing pages from memory when the prefetch feature in Suprtool is turned on.

Within Suprtool 4.5 and higher, you can turn this feature on with the MakeAbsent setting: Set MakeAbsent On.

This feature is only relevant if prefetch is turned on with a setting of 1-5, which controls the size of a prefetch. Set MakeAbsent On has no impact if prefetch is turned off.

The MakeAbsent setting was not documented, as we wanted to gain metrics on whether or not the setting was effective and under what cases this was effective.

What is Happening Now

As mentioned, we saw the problem as being long waits for disk IOs to be completed. This was primarily on non N-Class systems. Over time, the N-Class systems have been widely deployed in various customer sites and the symptoms that were observed were slightly different. Due to the dramatic improvement in disk IOs, the symptom we see on N-Class machines is a high utilization of memory. To get more information on N-Class machine disk IO, please see my paper “Living with 400 IOs Per Second” at www.robelle.com/library/papers/ios400.pdf

Some Metrics and Advice

So now that we have a general understanding of the symptoms and some of the settings that you can control to help alleviate the problem, let’s discuss how and what to change and when. Keep in mind that these are just general guidelines and your mileage may vary.

2Gb and less: For systems with less than 2Gb of memory you probably do not need to do anything. Of course, you are free to experiment with setting MakeAbsent on, or with changing the prefetch quantum or even turning it off. But the defaults should usually be fine.

Please note that when testing, your results may naturally improve if you repeat the same extract over and over, since more portions of the file or dataset will be in memory. You may want to make a change to the settings and monitor some jobs that you run frequently and at various times of the day, to see how the changes will impact your system over time.

My gut feeling overall for systems with less than 2Gb memory, the default setting of set prefetch 2 should be adequate and turning MakeAbsent on may hurt your system performance.

Greater than 8Gb: For systems with 8Gb of memory and larger, we have found that set makeabsent on has been helpful for memory consumption on N-Class systems and has reduced disc IO queues for non N-Class systems.

Between 2Gb and 8Gb: The results here have been mixed; as you get closer to 8 gigabytes turning MakeAbsent on has been better for N-class systems, but I have no real results for non N-class systems. For some systems turning prefetch off altogether proved to be the best option, but these systems were usually CPU bound. I would recommend first trying to turn off prefetch to see if that helps, and then turning it back on and trying MakeAbsent on. Again, I suggest doing this and leaving the setting for a day or two and measuring the results.

Should I change anything?

If you do not detect a problem, then normally I would not recommend changing anything blindly. Most people who call or e-mail about this issue have discovered it by using either Glance (from HP) or SOS (from Lund).

How do I make changes?

You can make global changes to these settings for every Suprtool process, by putting the various set commands in the file SUPRMGR.PUB.SYS

So to turn on the MakeAbsent feature, you could add set Makeabsent on to the suprmgr.pub.sys file. Of course if you see Set Prefetch 0 in the file, MakeAbsent will have no effect. If you do not see a Set Prefetch command, the default setting is Set Prefetch 2 and this is probably reasonable for most systems.

If you do experiment with this feature we would like to hear your results. Please send me an email at Neil@robelle.com.