Monday, May 29, 2017

Memorial Day 2017: Sully

One of my previous Memorial Day posts looked at a Clint Eastwood film about a veteran. Another Eastwood film conveniently found its way to my cable channel a few weeks go, also about a veteran: Sully. This film looks primarily at Chesley Sullenberger's time as a commercial airline pilot, centering on the mishap investigation around Sullenberger's decision to crash land his airliner in New York's Hudson river after a malfunction on January 15, 2009.

And once again, there are some parallels between this movie and the broken IT industry. The aviation world, once and arguably still one of the leaders in epitomizing the best of human endeavor, has grown large and complex, characterized by many silos of power. There are of course the regulatory commissions like the Federal Aviation Administration (FAA) and National Transportation Safety Board (NTSB), pilots' unions, air traffic controller unions, additional administrations at the state and municipal level, and because there wasn't enough fun coordinating all of that (and the multitude that I don't know about and didn't mention) organizations like the Transportation Security Administration (TSA) were added.

One of the key problems in IT departments now is that the fragmentation of duties creates complication and reduces efficiency. This isn't a revelation, it's only the same thing that has driven comedic material about organizations (especially governments) for decades, perhaps centuries. This derision is well earned, with governments often characterized as a circus, a whimsical collection of nincompoops that take up too much time and too much money. In an arena with many warriors it's easy for each member of a team to forget the overriding goal to serve a customer.

Each part of the equation is fighting to be a bigger part, to preserve itself, and to survive and thrive. This is natural behavior, but can lead groups in an organization astray. The focus becomes "me" instead of "we" and each faction serves itself and the spirit of the greater organization is lost.

In Sully, the NTSB is largely painted as an organization looking to place blame on the pilot for the mishap. Although it's not directly stated in the film, it's hard not to imagine external pressures on the NTSB to find fault with the pilot. We've seen this sort of thing before, when insurance companies and their lawyers would choose obfuscation over clarity, choose a pyrrhic victory at the expense of justice. The greater good it seems, isn't served by the initiative and actions of a pilot with some autonomy to save lives, but instead by entities uninvolved in the actual incident who would save corporate dollars.

Of course this is largely just me being cynical. I have to give Eastwood credit for not openly using such obvious gimmicks to win the audience to Sullenberger's case. No, there would instead be other gimmicks used to create drama where there might not have been any. I don't know exactly how the hearings went, but in Sully, there were a couple things I didn't like in this otherwise interesting and well made film. It was hard to believe that the late breaking evidence appearing in pivotal moments of the film mirrored the timing of those events of real life. I also found it a bit too easy for the NTSB to shift so quickly from villain to adoring admirer; in real life I'd hope the NTSB isn't there to create drama but simply to approach an investigation with no expectation that the cause is pilot error or technical malfunction and to be interested in only one thing: the truth.

Ideally, each organization involved would be singularly focused on that truth, the biggest one being that not a single life was lost in the incident. Isn't this the reason for all these entities, to push for safety and standards to limit this risk to human life? Without passengers, there's not much need for a smart, experienced pilot and co-pilot, safety boards, investigators, flight crew and their training, and air traffic controllers. Every single one of the people involved in those groups should take pride in ultimately serving the same customer: the passenger.

I wish it was that way in IT too. Whether it is a new software project to create a system that improves communication or efficiency, or a hardware team that is responsible for getting the appropriate technology into the hands of the right user, or a software support team, or a business analyst, they should all understand they have the same goal. They should not be looking to put down one team at the gain of their own, or to value performance metrics relevant only to their particular space, but to understand that the production experience with technology is everyone's responsibility.

That would be a great IT organization indeed. In that world, everyone works together to provide for the one true boss: the customer. In that world, everyone is doing their part to keep the customer happy even though each may have a different role. A circus, but the good kind.

Monday, May 30, 2016

Memorial Day 2016: A war story that predicted the future of IT

I like traditional paper books and resisted going to ebooks. I like to think it wasn't because I was stupid, I certainly understood the value of their portability but I thought ebooks were soulless. I grew up in a time where a quality bound book was a piece of education and also a work of art. It was collectible and you could get an author to sign it. It had value. But eventually I succumbed to the lure of being able to have thousands of books on my phone.

I was slow to adopt audio books too, but recently started up after flooding in Houston caused my commute to become longer. And I've wiped out three books in six weeks or so, improving my rate of attacking my backlog.

One of those books was Day of Infamy, by Walter Lord. It was written a long time ago but is a detailed overview of the Dec 7, 1941 Japanese attack on Pearl Harbor. It's a great book. The Pearl Harbor attack was a surprise and a shock to many on the islands. Some people even mistook the inbound Japanese airplanes for Americans on training maneuvers.

When the servicemen understood what was going on, many moved to respond, trying to help others or pass the word. Many others tried to put up whatever resistance they could, even firing at passing planes with small arms. Getting those small arms was sometimes difficult. Lord's book mentions a couple situations relevant to this blog.

There were some instances where even amid the attack the supply clerks refused to issue arms or ammunition to the men without the proper authorization. This represents several of the themes I've noticed in our IT industry.

  • Revering Process at the cost of Effectiveness. The intransigent supply clerks had a narrow focus on their role, which was essentially to control inventory. They lacked an appreciation for the larger picture and worshiped the traditions of their gods rather than the reality surrounding them. The impetus for this thinking may have been grounded in reasonable motives (cost control, safety, accountability) but such adherence to dogma rather than the primary goal of the institution (in the case of the US military, to defend the nation) may have cost lives. This anti-pattern is often the fallout of large organizations that have naturally segregated duties for specialization and formed silos of knowledge. These silos have their own management chains and can take a counterproductive focus as they work to justify their existence. This behavior doesn't require a large organization though; there are plenty of inexperienced managers out there that can do the same thing even in small companies.
  • The Road to Hell is Paved with Good Intentions. I probably don't need to explain this one. In a world increasingly driven by people and groups that have mastered the ability to push personal interests over what's really important, you can find plenty of real world examples.
  • The Tactical Reality will Override the Theoretical Ideal. Lord mentions that in some desperate cases, servicemen took axes to the locks on ammunition cages and did what they had to do. Yes, an accountant somewhere will be very hurt by the loss of the lock, but although Harlan Ellison astutely noted that "the world is becoming a cesspool of imbeciles," people are not completely stupid and sometimes humans can be surprisingly functional. Even though I can't stand the ludicrous edicts of the Sarbanes Oxley act, SOX procedures do allow for the people who can get the job done to have provisional authority in emergencies. But I'll bet money it wasn't Sarbanes or Oxley that allowed for that, but the grunts in the trenches that fought back against the original rules.
In honor of the servicemen that lost lives at Pearl Harbor that fateful day, I wish you a happy Memorial Day. 

Sunday, April 24, 2016

Software Lessons: The Customer is always right, and sometimes so are You

Here's one of my favorite memories from a project I worked on.

Our company had acquired another company and my team had the responsibility to integrate the new company's business into our system. There were some similarities between our existing business and the acquisition's business, but as there always are when companies merge, there were some critical differences too.

We worked really hard and managed to modify our system in ways that made the new company's work possible to do in our system. One minor feature they didn't ask for stands out in my memory. I added some auditing data to one of the screens we created for them. It would track simple things like when data was added or modified. I'd always found such things handy in support, so I did what I thought was the right thing and integrated it into the system in version 1, not as an afterthought hacked in two years later.

The analyst representing the acquisition noted in one of our design meetings that this was not a feature they needed or had asked for and why were we putting it in? I explained that it would be useful for ongoing support and that it would be done in a way that would not interfere with their work and that it would not negatively affect performance. The analyst still seemed a little bothered by this unasked for feature, but accepted it with my assurances. Note that by both some project management standards and agile development purists, this would have been considered a violation of the rules. In those worlds, I would have had to remove the feature (although it could arguably be a fair inclusion under what business analysts call a non-functional requirement).

Fast forward to about six weeks after the system went live. Things are going relatively well. The hard work in up-front design has mostly paid off and we had only couple notable bugs reported. Most either had workarounds or were fixed within the first post-implementation release. But then we get an interesting report from the same analyst that earlier had said there was no interest in the audit feature.

"We notice that the audit data doesn't seem to be working right. We expect the detail level items to be time stamped with the date of creation and they appear to be picking up the date of creation from the header level data instead."

I wanted to say, "Wait. Wow. Really? You are reporting a defect on a feature you told us you didn't want and wouldn't use?" But of course I said instead, "We can change that. We'll set up a change request and plan this for a future release."

Clearly, they'd been using the feature to see when data was created and/or modified and by whom. This is actually very useful reference information for both the support team and for the users. There are any number of situations where it's handy to know when and how something was changed, especially if that change is to foundational data that affects downstream processing. Unfortunately, not all systems track that information and many that do fail to make that information accessible.

The point here isn't to gloat that I was right. The point is that experience with support and systems can yield valid input into shaping subsequent systems. During design, there are many people of different experience levels and different personalities and perspectives trying to influence the product direction. Sometimes you, as the person that might live with the system after go-live, must find ways to get your ideas in place, even when traditional roles of expertise don't agree.

In theory, the International Institute of Business Analysis (IIBA) pays respects to the non-functional requirements, that is, requirements defined by the engineers. In reality, project managers and less worldly business analysts will overlook them in the interests of increasing project velocity. They don't have to support the beast for the next ten years, you do. So stand up for things that are reasonable and valuable for your team.

The customer is always right, and sometimes so are you.

Saturday, April 23, 2016

Software Lessons: The Long Tail

One of the top misconceptions I've witnessed in IT work is that many view a software project as a discrete component. They assume that the costs are static and once a piece of software is in place, the costs end or are minimal. They will look at the cost to build and implement on Day 1 as the only cost.

This is so wrong that it befuddles me how often this mistake is made. The "long tail" of a system's life, comprising the care and maintenance of the application, is likely to be far more expensive than the cost of the initial implementation (assuming the software is useful enough to live for more than a few years).

This is especially true if the quality of the application is poor due to corner-cutting in the creation process (and given the common emphasis of deadline over quality, it happens a lot). This is why developers and project managers are often at odds. The project manager is working to a milestone and then moving to the next project, but the developer likely has to live with the completed product for a longer duration, suffering with that application's deficiencies. I've discussed this before, where such scenarios create soul-sucking and inspiration-sapping work that is menial and tactical in nature rather than strategic. Many of the people at the high end of these mistakes are not held accountable for these problems, due to the way the typical reward system's metrics work.

Change will require a major shift in metrics, and it must come from the top.

Thursday, March 10, 2016

Shading rows for readability in a PowerBuilder datawindow

Just a little almost-useless tech note for myself.

I really like when developers do little things that make my life easier. Sometimes you'll see reports with a lot of data get shading to help your eye track the row you're reading.

I figured out a way to do this when I was doing a lot of PowerBuilder work. Apply an expression to the color property of the detail band of the datawindow. It's fairly easy to do if you just want to alternate every other row between two colors, but it gets trickier if you want to do it with more than odd or even rows.

Here's the code to shade every alternating set of three rows between light grey and white:
if ( mod(
          if( mod( getRow(), 3) <> 0
            , int(getRow() / 3)
            , int(getRow() / 3) - 1
         , 2
         ) = 0
   , rgb(255, 255, 255)   //white
   , rgb(233, 233, 233)   //grey

What this code is doing is using the modulus function to analyze the remainders of the row number divided by 3 to reduce it into one of two categories, grey or white.  In a division by three, the remainder would be 0, 1, or 2. If it's 1 or 2, we know it's the first or second row in a block of three rows. If it's a zero, we know it's a last row in a block of three rows.

But we want that third row to be included in the batch of three rows of a single color, and to alternate with every other set. So here's where the PowerBuilder INT function helps. We divide rows 1 and 2 by 3, and get a fraction, but the INT function simplifies them to the largest integer below the given value, so for rows 1 and 2, they INT function returns 0. But for row three we don't want it to fall into the next grouping so we subtract 1. Then we MOD these results by 2 to get the binary result of either a 0 or 1.

This makes our formula generate these values for the first several rows:

Row  Nested_IF  Outer_MOD
 1         0       0
 2         0       0
 3         0       0
 4         1       1
 5         1       1
 6         1       1
 7         2       0
 8         2       0

See how that works? It's fairly simple for the computer to process, but easily breaks each row down into either a 1 or a 0, and at that point it's pretty easy to wrap the whole thing in an IF statement and just assign the color.

I really only wrote this so I wouldn't forget it if I needed it again, but you're welcome to use it. If you're feeling "old school" you can use a light green instead of grey for the colored rows. And if you're a brainiac that figured out a way to do this with even less code, please post in the comments, I'd be happy to find an easier way.

Saturday, July 04, 2015

Book Review: A Technologist's Guide to Career Advancement by John Schneider

I saw someone mention this book in an article comment and bought it as it looked interesting. I also posted a review at Amazon but wanted to write a more detailed review here.

Advice from an IT Success

John Schneider has had an eventful and successful career in technology, working his way up to executive levels. This book couches itself as a career guide specifically for technology workers. I didn't find a whole lot that was specific to technology; there is a lot of great advice in this book for pretty much anyone. Schneider writes with an easy style and uses a lot of humor and the book is a fairly quick read. I think it's probably a good book for most people who want to know how to stand out in any industry, although it's likely a bit better value for younger people who have time to put into play the things Schneider recommends.

IT People Have an Edge?

The one conceit Schneider carries that's specific to tech workers is that you can do most of the other jobs in the company but the other people couldn't do yours.

This is often true but I would prefer that it not be presented as an absolute, because I've met my share of IT people that probably couldn't do other people's jobs, much less their own. And I've met smart folks outside IT that could be great in it.

But I do like the gist of what he's getting at because it matches an observation I've made about experienced IT teams: they represent an interesting foci of business and data. That is, they are people that understand the technology, but have also probably picked up knowledge of the company's business and its clients. They also are positioned to recognize the gaps between departments that need to use the same data. That puts them in a unique position to solve problems and improve the company.

As companies grow, they naturally tend to fragment and become a collection of silos, each narrowly focused on its specific function. This is a dangerous structure that compromises efficiency, ironically the very thing that specialization is supposed to provide. It becomes inefficient because people begin working in a vacuum and lose insight into why a task is done and how it affects others downstream in the process ecosystem. Communication tends to suffer too as people get busy doing their own things and this gradually reinforces both the distance between groups and the calcification of process quirks that might have been workarounds for something that was a problem once but might have since changed. And without an oversight to recognize this condition and an agent to promote improvement, companies (even if still solvent) eventually suffer atrophy. Teams become political and defensive, trying to justify their existence and role, even if they'd be better somewhere else.

The agents of change would ideally be managers and business analysts. But we know this isn't the case; higher management typically does not listen to the things their subordinates are telling them. They've evolved into selfish entities that cater to self preservation and shield themselves behind barriers of elitist cliques and faulty assumptions that they, and not the customer, are the profit center.

Killing Sacred Cows

Schneider isn't shy about challenging conventional career wisdom.

He disagrees with some things that are considered industry best practices, notably the advice on accepting a counter offer when resigning. The general rule is that you don't accept them. If there were fundamental problems at the job that caused you to want to resign, are they really going to change if you accept the counter offer? Schneider does acknowledge that if a workplace is really dreadful you should just leave, but then goes on to say the things you've heard about taking a counter offer are "BS" and taking one is just fine. I too will concede that if the parties involved are mature consenting adults and not children that can hold grudges, it might be ok.

But people are human beings and both the company, the managers, and your peers will remember what's transpired (you might try to keep it a secret, but things have a way of getting out). Ultimately, if you had to threaten to leave in order to get what you want, is that really the kind of place you want to stay? These are legitimate caveats to accepting a counter offer and Schneider is perhaps a bit to flippant about them; in addition when he calls them "BS" he doesn't really provide an argument about the specifics of why they are.

He also seems to value the effort one could put into acquiring certifications. I think a lot of experienced IT staffers will tell you different things here. In my career I've managed to do well without certifications. Schneider feels they will be useful in helping you advance in the organization and discounts the value of seniority; that may be true in some companies. But my personal experience is that certs are best as differentiators in getting hired, not promoted. Once you're in an organization, promotions are likely to be based on a combination of things such as performance, politics, and yes, seniority. Often, very much about politics and seniority.

Higher Education

He recommends getting an MBA; not bad advice, but there's more to this than meets the eye. He casually shoots down several excuses people might use to not get one when some of them are actually really good reasons. Cost, for example. If you want to go to a prestigious program, it might cost you the amount of a nice house; I would not so flippantly disregard this barrier. And, in my experience, the value of an MBA is like that of a cert. It's probably a great way to get your foot in the door, but advancing beyond that point will depend on performance, politics, and well, seniority, though I do see a distressing trend today to put young and inexperienced people into positions of power where they can destroy companies largely because they have an MBA and are experts at cost cutting, Mark Hurd-style. So maybe Schneider is right about that after all.

Are Things Different Now?

I don't doubt that Schneider is a successful and brilliant guy and probably a good boss too, if he practices what he preaches. However, I couldn't shake the feeling at times that he's led a bit of a charmed life. His thoughts about certs, seniority, and MBAs make me feel that he's been fortunate to traverse most of his career through meritocracies. But I'm certain I'm not alone as an IT staffer that's recognized technology workers have become the contemporary blue collar workers of the world. IT shops are seen as costs, not strategic components, by most companies. As a result IT people are constrained by a very thick ceiling barring them from the highest leadership positions (roles open to operations, sales, engineering, marketing, and even accounting) where they could bring their blend and breadth of business and systems knowledge together to truly help a company forge strategic initiatives in intelligent cost cutting rather than mere layoffs and the practice of being cheap at the expense of efficiency.

All this to say that while Schneider's advice is still overall very good, it may have had more effectiveness before IT departments evolved into the bastard stepchildren of a companies today, a time before PMP's started telling us to forego innovation for smaller and more easily measured changes. A time when workers were allowed to think.

A Word about Surveillance

Happy Fourth of July. I've got something to say about a controversial subject on a day the United States likes to reserve for a celebration of freedom.

It's common for ivory tower dwellers to scream about surveillance being a violation of privacy. And taken to pedantic extremes, they are correct. It certainly seems wrong to have your communications monitored and your actions tracked without your knowledge and consent.

People who have spent much of their lives in the ivory tower though are often challenged when it comes to separating fantasy from reality. The unfortunate reality is that humans are flawed; we will be dishonest for a number of reasons, we will succumb to base emotions, we allow hate and disrespect into our lives. We murder, steal, and cheat.

Perhaps that would be acceptable if our transgressions would hurt only ourselves but they don't. They infringe on the rights of others, and the idyllic image of a world totally "free" and totally "equal" is one that any reasonable and sane human understands is near impossible.

While it's easy to take the side of freedom in an argument, most people arguing for unbridled freedom are making assumptions that humans will always do the right thing. That assumption's been proven false since day one. It is more challenging to empathize with the other side. What would someone supporting surveillance say?

We can start with some true stories. I have a friend whose daughter lost her iPhone. But they'd installed a tracking app on it, and were able to find its location. My friend drove to the location, a residential address, and rang the doorbell. A young boy answered, and my friend said, "Call your parents over because I'd like to talk to them about the iPhone you stole." The boy was understandably shocked. When his mother called out, "Who's at the door," he responded with, "I've got it," and then surrendered the iPhone to my friend. Later, my friend proudly proclaimed, "I'm Batman!"

Here's another one. I recently was involved in an auto accident. A lady at a train crossing saw the crossing lights start to flash, and she braked abruptly, perhaps two to three car lengths ahead of the stopping line where you would expect stopping cars to line up. I jammed on my brakes in response, and was able to stop without hitting her. The young man following us was not able to stop and rear-ended my vehicle. The damage was mild and no one was hurt. We exchanged insurance information and I took several pictures of the scene, including a photo of the rear of the other vehicle. I did that to get the license plate, but it turns out it was a good idea because it shows no damage to the other vehicle's rear.

We then cleared the traffic pattern. The other driver seemed very nice and I didn't expect any problems. After filing a report with the other driver's insurance company, I found he lied about the incident, saying there was a third car involved that hit him from behind and pushed him into me. The insurance company stood by him because there was no other witness, even with my picture showing there was no impact damage to his vehicle. So to repair my vehicle I must now pay my deductible and my insurance company will pay the remainder. If the other guy had told the truth, everything would be covered by his insurance. What a pain in the ass.

The evidence I have should help my insurance company go after the other guy's insurance company and recover the costs, but why should they have to? Gee, you know what would have been really awesome here? A bit of footage from a traffic camera showing what happened. Oh, but that would have been more of the evil surveillance wouldn't it?

These are just simple personal anecdotes. You can find many stories of how traffic cameras and store cameras have helped uncover the truth about an incident where someone is lying. And when a criminal is caught, the people screaming for freedom from surveillance don't then go scream about the criminal being wronged, do they?

I sure hope they don't, because it means they do understand the challenge of navigating the line between freedom and security. They're quick to quote Benjamin Franklin, when Franklin said that a society willing to give up freedom for temporary security deserves neither freedom or security. But Mr. Franklin said that a LONG time ago and times have changed, people haven't. And old wily Ben Franklin was pretty smart...look carefully at the quote. He notes it is "temporary security" that a compromise of freedom gives, a fact that many drop from the quote when referring to it. So what would the price of more lasting security be?

Ask yourself this: at what point does a lack of security begin to infringe on freedom? If you cannot navigate in a society without your every step being compromised by the dishonest, how free are you? Where do you find fertile ground for enterprise? For family? The point I'm getting at here is that the people demanding total freedom incorrectly assume that freedom and security are mutually exclusive. Freedom and security are indeed strange partners, sometimes at odds but also sometimes allies. And when there's an altercation between them, how do we resolve it? Usually, we depend on that most neutral of third parties: the truth.

I'm not talking in absolutes; wisdom has taught me that a position of an absolute is usually wrong. And I remember my literature too, and would prefer not to be Winston Smith, screwing my girlfriend in a field while filmed by cameras in wheat stalks. But the world is an imperfect place, and in the battle to make it a better and more just one, the truth has a place.