The LEGO of System z Software (aka Pipelines)

As I look back over the things that impacted my productivity as a VM performance analyst and tester, I find myself coming back to CMS Pipelines. Read Melinda Varian's paper on the history of z/VM for more details on how IBM's John Hartmann came up with the idea, and how it found its way into the product.

I had heard tell of IBMers using Pipelines in late 1980s, but it was always bit-jockey types and I had assumed it was the same ones who got giddy over the idea of using a plugboard to program or using dials on control panels to set IPL addresses. In other words, people who were either way smarter than me or loved this stuff way more than me (or perhaps both). I didn't really start to embrace Pipelines until it was incorporated into VM/ESA V1R1.1 (CMS 8).

My love for Pipes grew exponential in that first year. At first, my reaction was, "This is interesting". Then I started to see what others were doing with it, and my reaction moved towards, "This is cool". Finally, as I began to use it for heavy lifting in my analysis work, "This is awesome".

I still remember trying to do some analysis to reduce some of VM/ESA's master processor usage in early 1990s, this was the time frame where larger n-ways were starting to appear on the horizon. The use of OV/VM was becoming heavier as well, and made many requests of CP that were master only related. It was an important area to improve. I started to think that to understand some of master only processing I would need to mine more data out of the CP trace table. I could gather large amounts of data through TRSAVE, but then what? This wasn't like dump analysis where the answer is typically within the last 200 trace entries. This was a place where Pipes became my friend as I would massage the data in various ways. Continuing to narrow in on the culprits.

This Pipes-aided analysis led to two changes in VM/ESA V1R2.1 that significantly reduced the master processor usage. That led to one of my first formal IBM awards. I'd like to thank the academy, my parents, and Pipes; without their support I would not have reached the solution as easily.

Pipes would come to the rescue at various times after that. I would write a little utility to read the CP trace table on a running systems and produce some stats that was useful for performance analysis in some extreme cases. The MONVIEW and Friends utilities now shipped as samples in z/VM are Pipes-based. Some of which actually involved user-written stages. At one point John even provided a Pipelines stage to directly read *Monitor data, appropriately named STARMON. Now I was giddy!

I never rose to Master Plumber status in the fraternity of Pipelines coders, but I loved them just the same. I admit playing with LEGO bricks also. The beauty is anyone can build what their mind sees. The child that creates a LEGO model car made up of 9 bricks and the man-child that builds a 10,000 brick replica of the United States capital building each go one brick at a time. And there would be much more stress if the bricks hadn't been designed to work together as they do. The possibilities are amazing. Just five 8-studded bricks of the same color can be put together in over 10 million configurations. Perhaps that's why no two Pipes programs are exactly alike.

I also just noticed. Both CMS Pipelines and LEGO bricks originated in Denmark. Keep plumbing and keep building.