Happy New Year (plus nerdiness)

Here's a post to prove I'm a nerd. If you haven't seen Firefly or Serenity yet, and you're an SF fan that got tired of the Star Trek formula, you might want to give it a shot. Then the rest of this post will make some sense.

-----

Your results:
You are Malcolm Reynolds (Captain)

Malcolm Reynolds (Captain) 70% Zoe Washburne (Second-in-command) 65% Wash (Ship Pilot) 65% Kaylee Frye (Ship Mechanic) 65% Dr. Simon Tam (Ship Medic) 55% Derrial Book (Shepherd) 40% Alliance 40% Inara Serra (Companion) 35% Jayne Cobb (Mercenary) 30% River (Stowaway) 20% A Reaver (Cannibal) 10%

Honest and a defender of the innocent. You sometimes make mistakes in judgment but you are generally good and would protect your crew from harm.

Click here to take the "Which Serenity character are you?" quiz...

More to come

I haven't been posting lately...getting settled into lots of new things. Had a job change just before the last post, and also on a bit of a different schedule now. But don't worry. I'm hardly out of ideas and rants.

Anyone can become a Developer; the Trick is staying a Developer

At the risk of alienating some of the readers, I will tell you about my favorite writer. It's this angry guy named Harlan Ellison. He's quite famous in the fiction circles, though it was his non-fiction that really won me. He has a cynical streak and is very intolerant of the incompetent. It's pretty refreshing stuff to read as you journey through this life and become frustrated with the reality that in the US, society not only tolerates mediocrity and incompetence, but encourages it.

I had the honor of hearing him speak at some of his public appearances. One of the things he said really stuck with me.

Anyone can become a writer. The trick is staying a writer.

I think that's a fascinating statement. To me, it means that each person knows at least one story that he or she can tell better than everyone else. But it also means that the art and profession of writing is something that requires more than the ability to communicate. It also requires some measure of passion to do well. There's another profession like that: software developer.

In an earlier post, I said that anyone could become a developer. Well, that's true, but I also said this:

Making software is easy, making it right is hard.

To really be outstanding at making something as complex as software, you need to have a true drive for learning and improvement. That means more than 40 hours/week of effort. That means taking time to read other blogs and books, and to take classes, even if the IT department you work in has cheap-ass management that doesn't understand that the best IT staff is one that's educated and skilled. That means paying for it yourself if you have to or finding work somewhere where the management values employee development.

This is especially true in an industry like IT, where the technologies can change in the space of a few years. When I first went to EDS, I learned COBOL. I didn't care for it, but it put food on the table. I always yearned to work on PC-based tools, and eventually got to a project where I learned PowerBuilder. But at this early stage, I was still struggling with learning so many things. I was still learning about my client's business, about event-driven programming, and about SQL, and I was clueless about Object Orientation. Frankly, I was not very good then, and my stuff sucked.

With time, I figured out the tool and now consider PowerBuilder to be one of the best development tools ever made. If you need to connect to and do work with an enterprise database (or even a less-than enterprise database) and you are not using the PowerBuilder datawindow, you are spending more money, time, and effort than you need to be.

So why did PowerBuilder (PB) lose it's luster in the market? A big part of that is the way Sybase handled the product after buying PowerSoft. But it didn't hurt that most developers are idiots. They are so into chasing after The Next Big Thing(TM) that they never stay with a language long enough to develop a level of mastery. I've seen a bunch of PB code over the years and most of it was mediocre at best. There are guys that have been using it for years that never truly utilized the power of the tool. It supported the big three of OO (encapsulation, inheritance, and polymorphism) long before Java ever existed! But guess how many of the PB apps that I've seen actually used OO? None, but in all fairness I didn't really understand it until I took a Steve Benfield class back in 1997. That is really sad, that most developers don't really use the power of OO (or PB), but a deeper discussion of that is for another day. To me, understanding OO is what truly separates the mediocre developers from those that finally hear the music and take that step towards advanced developer. Maybe I'm wrong, but the body of code I've seen indicates that many haven't made that leap.

Getting back to my point, one of the pivotal parts of staying a developer is to grow your knowledge. Besides commitment, patience is a big part of that. Experience counts for a lot in making software, because the core challenges in this industry aren't likely to change. Oh, the intensity and the trappings of the challenges can rise and fall with the changing times, but things like data access, data storage, security, concurrency, and performance will always be the things we struggle with, and the mediocre will fail to consider, happy as they are to simply toss out a huge nesting of IF statements. Striving for adaptability, agility, maintainability, and elegance are the things to which the mediocre will only be able to give lip service. I can't tell you the number of times I've seen complicated spaghetti code, and thought how it would be so much easier if the folks that built the app would have used true OO. OO is a bit tricky to figure out at first, but once you get there, you realize it's the only way to make code more agile.

Even after all the years of learning, I still consider myself far from where I'd like to be. But I see definite growth in my knowledge. And I did it mostly by focusing on one language. Thankfully, it was PB, and not something like a pre-.NET incarnation of Visual Basic. I was able to overcome the initial learning curve, which included the crucial step of making mistakes and learning from them, then continued to simultaneously grow my understanding of the advanced capabilities of the language while reading about programming theory in general. Little of this actually happened while scrambling to meet deadlines...most of it took place after-hours, reading technical journals or in specific technical classes.

That's why it kills me when companies get upset about the poor quality of software, but are at the same time throwing poorly managed software projects at their developers and refusing to pay the cost of employee development. If corporations don't want to invest in sending their janitors to advanced skills training, that's one thing, but if they don't send their thinkers to learn the new tricks in an industry where things are always changing, they're only hurting themselves. Still, if you're not one of the lucky ones working at Computerworld's Top 100 Companies to Work for in IT, then you're often like me, forced to figure out this stuff on your own.

That's why staying a developer is tough. It takes work to improve yourself. There are ways you can get away without putting in the extra effort, and there are also things that can compromise your efforts even if you do make them. More on that in a future rant. But if you're serious about staying in IT, at some point you're going to have to learn something either because you want to or because the environment forces you to. But don't always assume the learning has to be in a new language. If you're using the right language (something that supports OO like C, PB, Java, C#, or VB .NET), you'll be able to make dramatic improvements in your ability by studying advanced techniques and applying them right where you are.

This is where chasing The Next Big Thing can be wrong. I think that getting over the newness of a language is a critical part of being able to move on to using it with advanced concepts. If you don't get to the point where the syntax becomes second nature, then you'll always be in cut-and-paste mode, the thing that might be the biggest cause of PB's demise. Cut-and-paste is a symptom of procedural logic, and procedural logic is the mental baby food of poor or mediocre developers. I can guarantee you there are still people slapping procedural logic routines into their Java and .NET objects and then patting themselves on the back and saying, "Look! I did OO programming!" No, that's "Uh oh programming." The people that never figured out OO while learning PB took their same bad habits to Java, then to .NET, all in their pursuit of The Next Big Thing. The Next Big Thing can't materialize because it's always being compromised by The Worst Old Thing.

Don't be fooled by the few in this industry that write the magazine editorials and work in ivory towers where they get to play with whatever they want and don't have deadlines. They write about how your programming tool is dead and you are crap if you don't move to The Next Big Thing. I say to them, "You're lucky, good for you, now shut the hell up, not all jobs are like yours. The rest of us don't want to hear about your time at the tech conference in Hawaii. We have to figure out how to resolve problems in a myriad of apps supporting a multibillion dollar company, and no, the app isn't written in the latest and greatest, but it's putting food on my table. Who's going to do that if I take a few years off to study an obscure API in .NET? You?" Staying a developer requires learning, but learning isn't always about a new language, sometimes it's about making better what you have, and it should be more about improving productivity than improving resumes.

Once you've achieved proficiency with a language, and are confident that you understand OO and have migrated from cut-and-paste to model-design-and-iterate, then you're probably at the point where you can start picking up all kinds of newer languages because at least some of the universally useful concepts are understood and can be translated over. Sometimes, if you know what you're doing, you can appreciate when the new thing, say Java or .NET, can offer you something your old language didn't like implementation of operational polymorphism through things like interfaces. But I'll bet a lot of the ex-PB guys that went to Java and .NET first heard about interfaces and said, "Huh?" And if you've used interfaces for a while, you know they are powerful, but like recursion, can get confusing if you don't design them well and apply them judiciously. Guess who probably implemented hundreds of interfaces without thinking first?

Anyone can become a developer. But if you want to stay one, try climbing to the next level before moving to the next language. Try supporting your own crappy code for a while, feel the pain those that support and use your apps feel, learn from your mistakes, and realize that there are better ways of doing what you think you're so good at. If you don't, you'll be dumping more of your crap on the ones that follow, you'll just be doing it in a different language. You can count on the lack of metrics in our industry to shield you for a while, but sooner or later, the smart ones are going to stop hiring you.

Recent Outsourcing Horror Visions

I'm hardly the first person to gripe about this, but there's an unsettling trend in America to send work offshore. The proponents of this practice defend it by saying that manufacturing and industrial jobs also went overseas but didn't compromise American supremacy.

Well, they're half right. There are some responses to this trend in the July 2005 Software Development magazine's feedback section (registration required). Michael Wheelen's letter succinctly identifies the entrapment of the American IT worker. An excerpt:

"We as American workers are stuck. Until cost and pricing stabilize around the globe, our economy will suffer the most because we have the highest cost of living."

And:

"The global economy is here to stay, but let's replace the old dogs with young dogs - people who understand the new technology and can manage it to everyone's benefit; young blood can create a win-win for corporate America and the public. For example, why not let the corporations outsource, but when the end product returns to America, fix the profit to a certain percentage of cost?"

Great letter, but of course what Wheelen is asking for isn't going to happen. In particular, with software, there's no tangible product to tie a profit to, and as IT veterans know, no software truly has a finite cost - as long as it's alive and being used and maintained, the true total cost grows with the life of the software. The old dogs have a little too much power too, and it always seems the young dogs that finally do replace them have learned their tricks from the ones that went before.

In the same issue, another letter, American Decline, from Thomas Ronayne serves a chilling image. He wrote:

"American companies are run by fools interested only in short-term results, and the offshore community is ready, eager and willing to
step in and take over. Because the relationship between government and industry in the United States is adversarial (compared with that in, say, Japan and Korea), we've seen the steel industry more or less vanish, the automobile industry get into deep trouble and the electronics industry disappear...we're on the rapid road to becoming a client of other countries that are willing to invest in their future. If we had to fight a war with Asia, we wouldn't be able to produce the basic materials we'd need to do so."

I hope that's a touch more sensationalist than reality would have. I'd read not long ago about how the steel industry in Philadelphia is making a bit of a comeback because after demand outpaced supply in other countries, the US was able to compensate at a competitive rate. Economic equilibrium is the key to making globalization less painful. Still, I can see the US being stupid enough to get to a point someday where the native industrial capacity could be completely gone, or enough diminished that if a wily agent of chaos could organize our allies to turn on us, there would be no foundation to support a future generation's "Rosie the Riveter."

But in the previous generations of offshoring, and in Ronayne's vision, the jobs being sent away from North American borders were largely physical jobs not involving soft skills. And if you look at the industrial offshoring efforts, not all of the work even left North American borders, moving to Mexico and Canada (ah, how arrogant we are to assume that America or North America always means "the USA").

But the know-how is still in the US! If there were a disaster, or a war with large scale scope, I'm optimistic that our engineers and surveyors and construction workers could find a way to adjust and still get the work we needed done at home to support a war effort. And North America is still a gifted geographical wonder, with natural resources galore, and still bordered by large bodies of water. You can compromise your opponent's industrial capacity and maybe even its economy, but no one gets a free pass from Mother Nature. Invading the US wouldn't be the cakewalk that Ronayne thinks it would be.

Mind Your Contractors

I don't like the way many corporations use contract labor, especially in the IT market. I worked as a contractor and as a full-time employee, and my experience found that most contract programmers are mediocre. There are of course some good ones. The outstanding ones have a thorough understanding of the chosen tools and best practices, sport a respectable work ethic, deliver quality results (including documentation, for all of you guys out there that were about to say, "Oh, I'm one of the good ones!"), and are truly professionals. But most don't deliver on all those categories, and they have no impetus to change because companies aren't rating their work properly. Instead, some mediocre contractors are getting by because they have a good network or have been friendly with some managers. They are not necessarily good programmers, but they're good networkers.

Where to use Contractors

I'm not sure if I just bashed or praised the mediocre contractors, but in a perfect world, here's how I see it being done. Corporations should use contractors for certain specific needs:

• To shore up temporary staffing needs on a given project
• To temporarily apply a specific set of rare skills to a given project
• To use an expert for either of the above reasons, but also as a mentor to the staff for learning new techniques/technologies

The problem I saw in the 90's was that contractors were being hired at times almost like full-time employees. They were given project management responsibilities and often a fair amount of power. That's not necessarily wrong, if the contractor is good and is the type of person willing to contibute energy to a turnover phase (teaching about and documenting their work for the full-time staffer that takes over). But many times I saw mediocre or bad contractors tasked with important tasks like system design and development. Giving bad contractors important work sets up the hiring company for lots of downstream pain.

A developer may be superficially proficient in a given tool, but still not be what you'd call an advanced developer. Some could even be certified in the tool, but not have an understanding of related necessary knowledge, have a terrible work ethic, or be a poor communicator. In an earlier post, I lamented that many developers simply haven't mastered advanced concepts and techniques like object orientation, database modeling, agile development, and relational theory. I'm not a PhD in those topics either, but I'm always pushing to learn more and I know enough to know the difference between good and bad code when I see it. Contractors in the 90's made, and some lucky ones today still make, rates in excess of $70/hr, and for the ones that don't bring to the table the advanced skills I'm talking about, that's about $40/hr too much. Many contractors fall into this category of guys that can put together a basic GUI but not really think long term about the user and how they're going to operate this screen and what they could do to make the user's life easier. And it gets worse if you look below the surface. The mediocre developer designs poor table structures and once you've got a bad database foundation, you've set yourself up for endless hassles making the GUI and business logic have to dance around that. The point here is that most developers fall into this category, and therefore by the law of averages, so do most contractors. It's not an intentional malicious thing; the market paid well and needed guys, and companies were hiring, so it happened.

The Metric System

Part of the problem is a lack of metrics in our industry. It's hard to rate the quality of someone's code unless you have the technical ability to read and understand it, and the advanced skill in that tool set to contrast the best practice solution with what the contractor devises. There's no way to rate a developer's past work without access to their code. So most of the time, the interview is based on personality match and a basic resume bullet match. Some interviews may include a technical interview too, but these are also flawed, often testing the interviewee's ability to simply memorize a tool's help file, rather than asking about how they would apply techniques to a solution; such interviews can identify blatant resume liars, but are often as much to boost to the interviewer's ego than to really test the prospective hire's aptitude for quality IT work.

Ironically, a contractor possessing a good work ethic and communication skills but flawed or dated programming practices is just as dangerous or maybe more dangerous than the stronger developer that might not be as comfortable chatting with the CIO. Guess which one is more capable of grabbing management's ear? Of getting the new project work? Of then creating more excrement in the company's systems?

Just Rewards (for the foolish)

Here are the risks you face when hiring a sub-par contractor and giving that person important development work:

• Most developers, contractors or not, have an aversion to documenting their work. When the contractor leaves, you'd better have resources skilled enough to pick up the pieces they've left behind.
• If the contractor was brought in to build a new system, they've gained valuable business knowledge (or should have) as part of the analysis process. When they walk, so does that knowledge...often to a competitor who's hiring contractors to build a system and wants someone to have business knowledge on the resume. Also, the full-timer tasked with picking up the pieces has to go through the learning again because of the contractor's likely reluctance to document.
• If the contractor didn't employ good OO design, then the code will likely be full of hardcoding and old-fashioned procedural-style logic. That might have been ok in 1990, but by 2005, these guys should know better. Polymorphism is a very powerful tool in the hands of the right developer, and it is a good solution for addressing the many custom conditions modern systems need to handle, and the business rule changes certain to follow. Systems written by guys uninterested in educating themselves on newer techniques (techniques, not necessarily languages) will be hard to maintain and prone to heavy production support, which is a poor time sink for your full-timers left holding the bag.

"Let's build a crappy system!"

There are good contractors out there. As a former contractor myself, I strived to not produce the deficiencies I mention above. I always documented my work, often with formal manuals that could be passed on to those continuing my work. And I worked to apply best practices and think of the user when designing things. I made my share of mistakes, but by and large my clients were pleased. I enjoyed working as a contractor and getting to see a variety of projects and work with different people, and I might still be doing it if the market hadn't crashed. But as a full-timer now, I'm dealing with the things I mention above. The companies that hired the contractors weren't purposefully saying, "Let's build a crappy system!" but they had no way of knowing what was going on because they didn't have the ability to evaluate the true quality of the work.

Corporate America vs. The American Dream

It's sad to think about how the relationship between corporation and employee has changed in America. At one time, you rarely saw people switch jobs; they took care of their employer and the employer took care of them. There was a sense of duty and honor on both sides. Things are different in the present. It's common to see people switch jobs with great regularity.

What changed, and what caused such change? That's a question with perhaps many different answers. Corporate America will say employees are selfish and greedy and job hop to secure a better financial position. Workers will say that corporate America started it first, by laying off droves of employees while corporate heads got rich. Like him or not, if you've been in corporate America for any length of time since the 80's, you've got to have some appreciation for Michael Moore's sentiments when he refers to Roger Smith's move to offshoring in Roger and Me, sarcastically saying, "Roger Smith was a genius." I say to the pointy haired ones, "Yes, employees job hop to secure a better position, but that's better than doing it by selling your soul."

I'm not sure what side of the chicken and egg question comes out ahead in that argument. But I do know that the rich have been getting richer and the poor getting poorer for a long time. And it's a trend that needs to change before there is no middle class. I'm not an economics expert but it seems to me the middle class is the foundation for nearly every society we've seen. George Carlin joked that there are three classes: The rich make all the money, the middle class does all the work, and the poor scare the middle class into keeping its jobs. Carlin can be goofy at times, but in this case, he's right.

Do all the work! Is that all the middle class does? I said in an earlier post that you can't really be a good leader without some decent followers. Ever worked on a team where there were lots of people who thought they were king? I've seen it and it's not pretty. Too many chiefs and not enough indians, goes the saying, although perhaps the more politically correct version of that should say, "Too many Ghandis and not enough Patels." Damn it, someone has to do the work! We're a long way from the day when we have to worry about robots being able to make the human race obsolete, so that means there better be a middle class around to get things done. And when you think of America, after all, isn't the "work ethic" one of things that always comes to mind?

What would a world without a middle class look like? It depends on where you are. If you're one of the rich, it looks pretty damn good. You don't have to work and the masses will do anything for an opportunity to eat the crumbs off your seat cushion. But if the distribution stays the same, then it will mean that the rich will be outnumbered 9 to 1. Of course, the politicians will be on the side of the rich, so the poor won't be able to count on tanks and soldiers to support their thoughts of revolution.

What's the point? Just that perhaps for America to protect itself, it may have to alter the concept of the American Dream. That's an arrogant term anyway, since to improve and advance one's self isn't a goal indigenous to American soil. It should be termed the Human Dream, or perhaps the Human Aspiration. America was just fortunate it was in the right place at the right time, with the right people, and um, a powerful military, to make things happen. It became the geographical embodiment of this Human Aspiration. The land of opportunity.

But the world is changing. As more of the countries in what Dr. Thomas Barnett calls "the non-integrating gap" begin to come to grips with their disorder and civil wars, as more countries like India produce educated lower-cost alternatives to American labor, soon the "American Dream" will begin migrating to other parts of the world. Where before wars and political instability made countries infertile ground for the Dream, now there are more places where it can happen. The laws of supply and demand dictate that naturally the world's best and brightest won't need to make the pilgrimage to North America to make their lives better.

To combat this, perhaps America must make the transition from a country of great expectations to a country of great managed expectations. And this brings me back to corporate America, but also to the population as a whole. Our future leaders are going to have to convince people that exponential company growth and profits in an age of subdued American supremacy aren't realistic. They'll have to show that the Japanese have it right when the difference between the lowest worker's salary and higest executive's salary is a factor of 7, not 70 (and that number is even higher now in America - I'm quoting numbers from the 1980's). No human being is worth that much money. And not all the rich end up being philanthropists like Paul Allen, funding advances in aviation and spaceflight technology, or Bill Gates, funding biomedical research. Excessive money can sometimes be like excessive time; it finds not-so-nice places to trickle into, like politician wallets.

We need to learn that perhaps it's unrealistic to expect such gross gains from our stocks and even in our personal lifestyles. I know it sounds horribly un-American here, but how expensive would medical care costs be if doctors decided a Toyota Camry was good enough for them instead of a Mercedes? Maybe that's a bad example. But if everyone had more modest expectations (not that they gave up on improvement, but learned to not sacrifice all for strictly monetary gain) then perhaps you wouldn't see such greedy moves on the part of corporation heads. And perhaps their usual excuse, that they're doing it for stockholders, would stop holding water.

A Great Cautionary Tale

I don't know why I didn't think of this the first time I saw Lord of the Rings: Return of the King, but seeing it again reminded me of a cautionary lesson everyone needs to know. Not just IT, but America as a whole. There's this wonderful scene at the end where Aragorn as the king sees the Hobbits pay their respects to him by bowing. He tells them, "My friends, you bow to no one," and everyone there, the king, the soldiers, the heroes of the dwarves and the elves, all bow to the Hobbits.

If I'm remembering correctly, this was already explained in professional criticism of Tolkien's work as his warning that mighty heads of state should not forget the little people that make up their constituency. This is hardly a ground-breaking moral. Leaders have been compromising their oaths for personal gain since, well, probably since Tomak the caveman leader first clubbed someone else's wife over the head because he was tired of his own. As humans, we've proven time and again that we just can't control ourselves. We've really got to work on that.

But without the people doing a leader's bidding, we would be nowhere. I'd give our societies less than a week to dissolve into chaos if the little people in charge of getting water to our homes decided to quit. Or less, if they also cut off sewage operation.

The moral is simple for world leaders: You cannot be a leader without reliable followers. It's also obvious for businessmen. There's always much talk about how to be a good manager but if no one follows, even a good manager is useless. Any manager can look good if he has great resources. But the greatest managers do it facing compromised resources and unexpected challenges.