Monday, May 30, 2016

Memorial Day 2016: A war story that predicted the future of IT

I like traditional paper books and resisted going to ebooks. I like to think it wasn't because I was stupid, I certainly understood the value of their portability but I thought ebooks were soulless. I grew up in a time where a quality bound book was a piece of education and also a work of art. It was collectible and you could get an author to sign it. It had value. But eventually I succumbed to the lure of being able to have thousands of books on my phone.

I was slow to adopt audio books too, but recently started up after flooding in Houston caused my commute to become longer. And I've wiped out three books in six weeks or so, improving my rate of attacking my backlog.

One of those books was Day of Infamy, by Walter Lord. It was written a long time ago but is a detailed overview of the Dec 7, 1941 Japanese attack on Pearl Harbor. It's a great book. The Pearl Harbor attack was a surprise and a shock to many on the islands. Some people even mistook the inbound Japanese airplanes for Americans on training maneuvers.

When the servicemen understood what was going on, many moved to respond, trying to help others or pass the word. Many others tried to put up whatever resistance they could, even firing at passing planes with small arms. Getting those small arms was sometimes difficult. Lord's book mentions a couple situations relevant to this blog.

There were some instances where even amid the attack the supply clerks refused to issue arms or ammunition to the men without the proper authorization. This represents several of the themes I've noticed in our IT industry.

  • Revering Process at the cost of Effectiveness. The intransigent supply clerks had a narrow focus on their role, which was essentially to control inventory. They lacked an appreciation for the larger picture and worshiped the traditions of their gods rather than the reality surrounding them. The impetus for this thinking may have been grounded in reasonable motives (cost control, safety, accountability) but such adherence to dogma rather than the primary goal of the institution (in the case of the US military, to defend the nation) may have cost lives. This anti-pattern is often the fallout of large organizations that have naturally segregated duties for specialization and formed silos of knowledge. These silos have their own management chains and can take a counterproductive focus as they work to justify their existence. This behavior doesn't require a large organization though; there are plenty of inexperienced managers out there that can do the same thing even in small companies.
  • The Road to Hell is Paved with Good Intentions. I probably don't need to explain this one. In a world increasingly driven by people and groups that have mastered the ability to push personal interests over what's really important, you can find plenty of real world examples.
  • The Tactical Reality will Override the Theoretical Ideal. Lord mentions that in some desperate cases, servicemen took axes to the locks on ammunition cages and did what they had to do. Yes, an accountant somewhere will be very hurt by the loss of the lock, but although Harlan Ellison astutely noted that "the world is becoming a cesspool of imbeciles," people are not completely stupid and sometimes humans can be surprisingly functional. Even though I can't stand the ludicrous edicts of the Sarbanes Oxley act, SOX procedures do allow for the people who can get the job done to have provisional authority in emergencies. But I'll bet money it wasn't Sarbanes or Oxley that allowed for that, but the grunts in the trenches that fought back against the original rules.
In honor of the servicemen that lost lives at Pearl Harbor that fateful day, I wish you a happy Memorial Day. 

Sunday, April 24, 2016

Software Lessons: The Customer is always right, and sometimes so are You

Here's one of my favorite memories from a project I worked on.

Our company had acquired another company and my team had the responsibility to integrate the new company's business into our system. There were some similarities between our existing business and the acquisition's business, but as there always are when companies merge, there were some critical differences too.

We worked really hard and managed to modify our system in ways that made the new company's work possible to do in our system. One minor feature they didn't ask for stands out in my memory. I added some auditing data to one of the screens we created for them. It would track simple things like when data was added or modified. I'd always found such things handy in support, so I did what I thought was the right thing and integrated it into the system in version 1, not as an afterthought hacked in two years later.

The analyst representing the acquisition noted in one of our design meetings that this was not a feature they needed or had asked for and why were we putting it in? I explained that it would be useful for ongoing support and that it would be done in a way that would not interfere with their work and that it would not negatively affect performance. The analyst still seemed a little bothered by this unasked for feature, but accepted it with my assurances. Note that by both some project management standards and agile development purists, this would have been considered a violation of the rules. In those worlds, I would have had to remove the feature (although it could arguably be a fair inclusion under what business analysts call a non-functional requirement).

Fast forward to about six weeks after the system went live. Things are going relatively well. The hard work in up-front design has mostly paid off and we had only couple notable bugs reported. Most either had workarounds or were fixed within the first post-implementation release. But then we get an interesting report from the same analyst that earlier had said there was no interest in the audit feature.

"We notice that the audit data doesn't seem to be working right. We expect the detail level items to be time stamped with the date of creation and they appear to be picking up the date of creation from the header level data instead."

I wanted to say, "Wait. Wow. Really? You are reporting a defect on a feature you told us you didn't want and wouldn't use?" But of course I said instead, "We can change that. We'll set up a change request and plan this for a future release."

Clearly, they'd been using the feature to see when data was created and/or modified and by whom. This is actually very useful reference information for both the support team and for the users. There are any number of situations where it's handy to know when and how something was changed, especially if that change is to foundational data that affects downstream processing. Unfortunately, not all systems track that information and many that do fail to make that information accessible.

The point here isn't to gloat that I was right. The point is that experience with support and systems can yield valid input into shaping subsequent systems. During design, there are many people of different experience levels and different personalities and perspectives trying to influence the product direction. Sometimes you, as the person that might live with the system after go-live, must find ways to get your ideas in place, even when traditional roles of expertise don't agree.

In theory, the International Institute of Business Analysis (IIBA) pays respects to the non-functional requirements, that is, requirements defined by the engineers. In reality, project managers and less worldly business analysts will overlook them in the interests of increasing project velocity. They don't have to support the beast for the next ten years, you do. So stand up for things that are reasonable and valuable for your team.

The customer is always right, and sometimes so are you.

Saturday, April 23, 2016

Software Lessons: The Long Tail

One of the top misconceptions I've witnessed in IT work is that many view a software project as a discrete component. They assume that the costs are static and once a piece of software is in place, the costs end or are minimal. They will look at the cost to build and implement on Day 1 as the only cost.

This is so wrong that it befuddles me how often this mistake is made. The "long tail" of a system's life, comprising the care and maintenance of the application, is likely to be far more expensive than the cost of the initial implementation (assuming the software is useful enough to live for more than a few years).

This is especially true if the quality of the application is poor due to corner-cutting in the creation process (and given the common emphasis of deadline over quality, it happens a lot). This is why developers and project managers are often at odds. The project manager is working to a milestone and then moving to the next project, but the developer likely has to live with the completed product for a longer duration, suffering with that application's deficiencies. I've discussed this before, where such scenarios create soul-sucking and inspiration-sapping work that is menial and tactical in nature rather than strategic. Many of the people at the high end of these mistakes are not held accountable for these problems, due to the way the typical reward system's metrics work.

Change will require a major shift in metrics, and it must come from the top.

Thursday, March 10, 2016

Shading rows for readability in a PowerBuilder datawindow

Just a little almost-useless tech note for myself.

I really like when developers do little things that make my life easier. Sometimes you'll see reports with a lot of data get shading to help your eye track the row you're reading.

I figured out a way to do this when I was doing a lot of PowerBuilder work. Apply an expression to the color property of the detail band of the datawindow. It's fairly easy to do if you just want to alternate every other row between two colors, but it gets trickier if you want to do it with more than odd or even rows.

Here's the code to shade every alternating set of three rows between light grey and white:
if ( mod(
          if( mod( getRow(), 3) <> 0
            , int(getRow() / 3)
            , int(getRow() / 3) - 1
         , 2
         ) = 0
   , rgb(255, 255, 255)   //white
   , rgb(233, 233, 233)   //grey

What this code is doing is using the modulus function to analyze the remainders of the row number divided by 3 to reduce it into one of two categories, grey or white.  In a division by three, the remainder would be 0, 1, or 2. If it's 1 or 2, we know it's the first or second row in a block of three rows. If it's a zero, we know it's a last row in a block of three rows.

But we want that third row to be included in the batch of three rows of a single color, and to alternate with every other set. So here's where the PowerBuilder INT function helps. We divide rows 1 and 2 by 3, and get a fraction, but the INT function simplifies them to the largest integer below the given value, so for rows 1 and 2, they INT function returns 0. But for row three we don't want it to fall into the next grouping so we subtract 1. Then we MOD these results by 2 to get the binary result of either a 0 or 1.

This makes our formula generate these values for the first several rows:

Row  Nested_IF  Outer_MOD
 1         0       0
 2         0       0
 3         0       0
 4         1       1
 5         1       1
 6         1       1
 7         2       0
 8         2       0

See how that works? It's fairly simple for the computer to process, but easily breaks each row down into either a 1 or a 0, and at that point it's pretty easy to wrap the whole thing in an IF statement and just assign the color.

I really only wrote this so I wouldn't forget it if I needed it again, but you're welcome to use it. If you're feeling "old school" you can use a light green instead of grey for the colored rows. And if you're a brainiac that figured out a way to do this with even less code, please post in the comments, I'd be happy to find an easier way.

Saturday, July 04, 2015

Book Review: A Technologist's Guide to Career Advancement by John Schneider

I saw someone mention this book in an article comment and bought it as it looked interesting. I also posted a review at Amazon but wanted to write a more detailed review here.

Advice from an IT Success

John Schneider has had an eventful and successful career in technology, working his way up to executive levels. This book couches itself as a career guide specifically for technology workers. I didn't find a whole lot that was specific to technology; there is a lot of great advice in this book for pretty much anyone. Schneider writes with an easy style and uses a lot of humor and the book is a fairly quick read. I think it's probably a good book for most people who want to know how to stand out in any industry, although it's likely a bit better value for younger people who have time to put into play the things Schneider recommends.

IT People Have an Edge?

The one conceit Schneider carries that's specific to tech workers is that you can do most of the other jobs in the company but the other people couldn't do yours.

This is often true but I would prefer that it not be presented as an absolute, because I've met my share of IT people that probably couldn't do other people's jobs, much less their own. And I've met smart folks outside IT that could be great in it.

But I do like the gist of what he's getting at because it matches an observation I've made about experienced IT teams: they represent an interesting foci of business and data. That is, they are people that understand the technology, but have also probably picked up knowledge of the company's business and its clients. They also are positioned to recognize the gaps between departments that need to use the same data. That puts them in a unique position to solve problems and improve the company.

As companies grow, they naturally tend to fragment and become a collection of silos, each narrowly focused on its specific function. This is a dangerous structure that compromises efficiency, ironically the very thing that specialization is supposed to provide. It becomes inefficient because people begin working in a vacuum and lose insight into why a task is done and how it affects others downstream in the process ecosystem. Communication tends to suffer too as people get busy doing their own things and this gradually reinforces both the distance between groups and the calcification of process quirks that might have been workarounds for something that was a problem once but might have since changed. And without an oversight to recognize this condition and an agent to promote improvement, companies (even if still solvent) eventually suffer atrophy. Teams become political and defensive, trying to justify their existence and role, even if they'd be better somewhere else.

The agents of change would ideally be managers and business analysts. But we know this isn't the case; higher management typically does not listen to the things their subordinates are telling them. They've evolved into selfish entities that cater to self preservation and shield themselves behind barriers of elitist cliques and faulty assumptions that they, and not the customer, are the profit center.

Killing Sacred Cows

Schneider isn't shy about challenging conventional career wisdom.

He disagrees with some things that are considered industry best practices, notably the advice on accepting a counter offer when resigning. The general rule is that you don't accept them. If there were fundamental problems at the job that caused you to want to resign, are they really going to change if you accept the counter offer? Schneider does acknowledge that if a workplace is really dreadful you should just leave, but then goes on to say the things you've heard about taking a counter offer are "BS" and taking one is just fine. I too will concede that if the parties involved are mature consenting adults and not children that can hold grudges, it might be ok.

But people are human beings and both the company, the managers, and your peers will remember what's transpired (you might try to keep it a secret, but things have a way of getting out). Ultimately, if you had to threaten to leave in order to get what you want, is that really the kind of place you want to stay? These are legitimate caveats to accepting a counter offer and Schneider is perhaps a bit to flippant about them; in addition when he calls them "BS" he doesn't really provide an argument about the specifics of why they are.

He also seems to value the effort one could put into acquiring certifications. I think a lot of experienced IT staffers will tell you different things here. In my career I've managed to do well without certifications. Schneider feels they will be useful in helping you advance in the organization and discounts the value of seniority; that may be true in some companies. But my personal experience is that certs are best as differentiators in getting hired, not promoted. Once you're in an organization, promotions are likely to be based on a combination of things such as performance, politics, and yes, seniority. Often, very much about politics and seniority.

Higher Education

He recommends getting an MBA; not bad advice, but there's more to this than meets the eye. He casually shoots down several excuses people might use to not get one when some of them are actually really good reasons. Cost, for example. If you want to go to a prestigious program, it might cost you the amount of a nice house; I would not so flippantly disregard this barrier. And, in my experience, the value of an MBA is like that of a cert. It's probably a great way to get your foot in the door, but advancing beyond that point will depend on performance, politics, and well, seniority, though I do see a distressing trend today to put young and inexperienced people into positions of power where they can destroy companies largely because they have an MBA and are experts at cost cutting, Mark Hurd-style. So maybe Schneider is right about that after all.

Are Things Different Now?

I don't doubt that Schneider is a successful and brilliant guy and probably a good boss too, if he practices what he preaches. However, I couldn't shake the feeling at times that he's led a bit of a charmed life. His thoughts about certs, seniority, and MBAs make me feel that he's been fortunate to traverse most of his career through meritocracies. But I'm certain I'm not alone as an IT staffer that's recognized technology workers have become the contemporary blue collar workers of the world. IT shops are seen as costs, not strategic components, by most companies. As a result IT people are constrained by a very thick ceiling barring them from the highest leadership positions (roles open to operations, sales, engineering, marketing, and even accounting) where they could bring their blend and breadth of business and systems knowledge together to truly help a company forge strategic initiatives in intelligent cost cutting rather than mere layoffs and the practice of being cheap at the expense of efficiency.

All this to say that while Schneider's advice is still overall very good, it may have had more effectiveness before IT departments evolved into the bastard stepchildren of a companies today, a time before PMP's started telling us to forego innovation for smaller and more easily measured changes. A time when workers were allowed to think.

A Word about Surveillance

Happy Fourth of July. I've got something to say about a controversial subject on a day the United States likes to reserve for a celebration of freedom.

It's common for ivory tower dwellers to scream about surveillance being a violation of privacy. And taken to pedantic extremes, they are correct. It certainly seems wrong to have your communications monitored and your actions tracked without your knowledge and consent.

People who have spent much of their lives in the ivory tower though are often challenged when it comes to separating fantasy from reality. The unfortunate reality is that humans are flawed; we will be dishonest for a number of reasons, we will succumb to base emotions, we allow hate and disrespect into our lives. We murder, steal, and cheat.

Perhaps that would be acceptable if our transgressions would hurt only ourselves but they don't. They infringe on the rights of others, and the idyllic image of a world totally "free" and totally "equal" is one that any reasonable and sane human understands is near impossible.

While it's easy to take the side of freedom in an argument, most people arguing for unbridled freedom are making assumptions that humans will always do the right thing. That assumption's been proven false since day one. It is more challenging to empathize with the other side. What would someone supporting surveillance say?

We can start with some true stories. I have a friend whose daughter lost her iPhone. But they'd installed a tracking app on it, and were able to find its location. My friend drove to the location, a residential address, and rang the doorbell. A young boy answered, and my friend said, "Call your parents over because I'd like to talk to them about the iPhone you stole." The boy was understandably shocked. When his mother called out, "Who's at the door," he responded with, "I've got it," and then surrendered the iPhone to my friend. Later, my friend proudly proclaimed, "I'm Batman!"

Here's another one. I recently was involved in an auto accident. A lady at a train crossing saw the crossing lights start to flash, and she braked abruptly, perhaps two to three car lengths ahead of the stopping line where you would expect stopping cars to line up. I jammed on my brakes in response, and was able to stop without hitting her. The young man following us was not able to stop and rear-ended my vehicle. The damage was mild and no one was hurt. We exchanged insurance information and I took several pictures of the scene, including a photo of the rear of the other vehicle. I did that to get the license plate, but it turns out it was a good idea because it shows no damage to the other vehicle's rear.

We then cleared the traffic pattern. The other driver seemed very nice and I didn't expect any problems. After filing a report with the other driver's insurance company, I found he lied about the incident, saying there was a third car involved that hit him from behind and pushed him into me. The insurance company stood by him because there was no other witness, even with my picture showing there was no impact damage to his vehicle. So to repair my vehicle I must now pay my deductible and my insurance company will pay the remainder. If the other guy had told the truth, everything would be covered by his insurance. What a pain in the ass.

The evidence I have should help my insurance company go after the other guy's insurance company and recover the costs, but why should they have to? Gee, you know what would have been really awesome here? A bit of footage from a traffic camera showing what happened. Oh, but that would have been more of the evil surveillance wouldn't it?

These are just simple personal anecdotes. You can find many stories of how traffic cameras and store cameras have helped uncover the truth about an incident where someone is lying. And when a criminal is caught, the people screaming for freedom from surveillance don't then go scream about the criminal being wronged, do they?

I sure hope they don't, because it means they do understand the challenge of navigating the line between freedom and security. They're quick to quote Benjamin Franklin, when Franklin said that a society willing to give up freedom for temporary security deserves neither freedom or security. But Mr. Franklin said that a LONG time ago and times have changed, people haven't. And old wily Ben Franklin was pretty smart...look carefully at the quote. He notes it is "temporary security" that a compromise of freedom gives, a fact that many drop from the quote when referring to it. So what would the price of more lasting security be?

Ask yourself this: at what point does a lack of security begin to infringe on freedom? If you cannot navigate in a society without your every step being compromised by the dishonest, how free are you? Where do you find fertile ground for enterprise? For family? The point I'm getting at here is that the people demanding total freedom incorrectly assume that freedom and security are mutually exclusive. Freedom and security are indeed strange partners, sometimes at odds but also sometimes allies. And when there's an altercation between them, how do we resolve it? Usually, we depend on that most neutral of third parties: the truth.

I'm not talking in absolutes; wisdom has taught me that a position of an absolute is usually wrong. And I remember my literature too, and would prefer not to be Winston Smith, screwing my girlfriend in a field while filmed by cameras in wheat stalks. But the world is an imperfect place, and in the battle to make it a better and more just one, the truth has a place.

Monday, May 25, 2015

Memorial Day Post 2015

Two years ago I posted after reading Chris Kyle's American Sniper. Today I finished watching the Clint Eastwood-directed movie. Today I'll share a few thoughts.

First, there's no way a two hour movie can fully capture the thoughts in the book. And it's rare when a movie captures well the spirit of the source material. The movie here is a heavily abbreviated version of the book's story. It also takes the usual artistic license to embellish the drama, or outright create it where there might not have been any.

Still, Eastwood made this movie knowing things Kyle could not have at the time Kyle wrote his book. The most important one being that Kyle's life would end in early 2013. And this is a powerful tool in story telling. Some writers will tell you that the hardest things to write are the beginnings and endings of a story; with the death of Kyle, Eastwood got built-in ending, and I think he uses it to good effect. Where the book was a slice of Kyle's life, describing some of his youth and life in the military, Eastwood's movie seems more a cautionary tale about war and the different ways it can effect people.

The combat scenes are good, except that I don't remember them being described in much detail in Kyle's book. But it's when Eastwood moves away from the book as scaffolding and fills in the blanks where it picks up it's best points. The movie does a good job replicating the eerie intimacy a sniper has with his targets, being farther away from them yet able to see more through scopes than the troops that are closer.

And when the actors are showing Kyle's estrangement from civilian life when he returns home, they're giving us something Kyle, who rarely comes across in his book as anything other than calm and in control, could probably not share as well as his wife could.

There's also something clever done as each of Kyle's tours is introduced. The equipment shown evolves with the passage of time. Early on we see Hummers, by the end, we see MRAP (mine-resistant, ambush protected) trucks and drones.

Unfortunately, it's when the movie casually tries to keep itself tied to the book that it falls. Kyle wrote with conviction about his friendships and his love for his fellow soldiers. The movie doesn't really bring into focus any of the peripheral characters. Even if they weren't made of cardboard we see so little of them that we don't really develop any affinity. Some film critics felt this same way about Kyle's wife, Taya, but I thought the actress did pretty well here with the moments she was given.

Aside from the biographical value of the film, I think the best message to be taken from the film is that war affects people in many ways. Sebastian Junger's book, War, also talks about this. Although PTSD is a more common term now, we're still learning a lot about it and how to deal with it.

One of the reviews I read said that although the film, like Kyle's book, is wrapped largely in jingoistic gloss, it is slyly also a an anti-war movie. "Slyly"? If you need a movie to convince you war is a terrible thing, you're an idiot. But aside from that, there are indeed a couple moments where I thought Eastwood challenged some of Kyle's beliefs. In his book, Kyle is proud to be a redneck of the "god, country, and family" philosophy, and this is mentioned too in the movie. But there is a quick scene where a more pensive teammate questions the US efforts. And I like the way it was portrayed; the actor doesn't spout off some speech written by some liberal ivory tower resident. Instead, when Kyle tells him the enemy is savage and evil, he simply responds with a non-committal, "Ok." And the delivery of his response was perfect, seeing it in print can't do it justice. It was said in a resigned tone, as if he knew he wasn't going to change Kyle's mind but also that while some of what Kyle said was true he had doubts about the US approach.

The movie finishes on the note that it should. In the same way that Unforgiven refused to revel in the act of killing, American Sniper ends quietly, with images that are respectful and yet deeply somber.