Happy New Year (plus nerdiness)

Here's a post to prove I'm a nerd. If you haven't seen Firefly or Serenity yet, and you're an SF fan that got tired of the Star Trek formula, you might want to give it a shot. Then the rest of this post will make some sense.

-----

Your results:
You are Malcolm Reynolds (Captain)

Malcolm Reynolds (Captain) 70% Zoe Washburne (Second-in-command) 65% Wash (Ship Pilot) 65% Kaylee Frye (Ship Mechanic) 65% Dr. Simon Tam (Ship Medic) 55% Derrial Book (Shepherd) 40% Alliance 40% Inara Serra (Companion) 35% Jayne Cobb (Mercenary) 30% River (Stowaway) 20% A Reaver (Cannibal) 10%

Honest and a defender of the innocent. You sometimes make mistakes in judgment but you are generally good and would protect your crew from harm.

Click here to take the "Which Serenity character are you?" quiz...

More to come

I haven't been posting lately...getting settled into lots of new things. Had a job change just before the last post, and also on a bit of a different schedule now. But don't worry. I'm hardly out of ideas and rants.

Anyone can become a Developer; the Trick is staying a Developer

At the risk of alienating some of the readers, I will tell you about my favorite writer. It's this angry guy named Harlan Ellison. He's quite famous in the fiction circles, though it was his non-fiction that really won me. He has a cynical streak and is very intolerant of the incompetent. It's pretty refreshing stuff to read as you journey through this life and become frustrated with the reality that in the US, society not only tolerates mediocrity and incompetence, but encourages it.

I had the honor of hearing him speak at some of his public appearances. One of the things he said really stuck with me.

Anyone can become a writer. The trick is staying a writer.

I think that's a fascinating statement. To me, it means that each person knows at least one story that he or she can tell better than everyone else. But it also means that the art and profession of writing is something that requires more than the ability to communicate. It also requires some measure of passion to do well. There's another profession like that: software developer.

In an earlier post, I said that anyone could become a developer. Well, that's true, but I also said this:

Making software is easy, making it right is hard.

To really be outstanding at making something as complex as software, you need to have a true drive for learning and improvement. That means more than 40 hours/week of effort. That means taking time to read other blogs and books, and to take classes, even if the IT department you work in has cheap-ass management that doesn't understand that the best IT staff is one that's educated and skilled. That means paying for it yourself if you have to or finding work somewhere where the management values employee development.

This is especially true in an industry like IT, where the technologies can change in the space of a few years. When I first went to EDS, I learned COBOL. I didn't care for it, but it put food on the table. I always yearned to work on PC-based tools, and eventually got to a project where I learned PowerBuilder. But at this early stage, I was still struggling with learning so many things. I was still learning about my client's business, about event-driven programming, and about SQL, and I was clueless about Object Orientation. Frankly, I was not very good then, and my stuff sucked.

With time, I figured out the tool and now consider PowerBuilder to be one of the best development tools ever made. If you need to connect to and do work with an enterprise database (or even a less-than enterprise database) and you are not using the PowerBuilder datawindow, you are spending more money, time, and effort than you need to be.

So why did PowerBuilder (PB) lose it's luster in the market? A big part of that is the way Sybase handled the product after buying PowerSoft. But it didn't hurt that most developers are idiots. They are so into chasing after The Next Big Thing(TM) that they never stay with a language long enough to develop a level of mastery. I've seen a bunch of PB code over the years and most of it was mediocre at best. There are guys that have been using it for years that never truly utilized the power of the tool. It supported the big three of OO (encapsulation, inheritance, and polymorphism) long before Java ever existed! But guess how many of the PB apps that I've seen actually used OO? None, but in all fairness I didn't really understand it until I took a Steve Benfield class back in 1997. That is really sad, that most developers don't really use the power of OO (or PB), but a deeper discussion of that is for another day. To me, understanding OO is what truly separates the mediocre developers from those that finally hear the music and take that step towards advanced developer. Maybe I'm wrong, but the body of code I've seen indicates that many haven't made that leap.

Getting back to my point, one of the pivotal parts of staying a developer is to grow your knowledge. Besides commitment, patience is a big part of that. Experience counts for a lot in making software, because the core challenges in this industry aren't likely to change. Oh, the intensity and the trappings of the challenges can rise and fall with the changing times, but things like data access, data storage, security, concurrency, and performance will always be the things we struggle with, and the mediocre will fail to consider, happy as they are to simply toss out a huge nesting of IF statements. Striving for adaptability, agility, maintainability, and elegance are the things to which the mediocre will only be able to give lip service. I can't tell you the number of times I've seen complicated spaghetti code, and thought how it would be so much easier if the folks that built the app would have used true OO. OO is a bit tricky to figure out at first, but once you get there, you realize it's the only way to make code more agile.

Even after all the years of learning, I still consider myself far from where I'd like to be. But I see definite growth in my knowledge. And I did it mostly by focusing on one language. Thankfully, it was PB, and not something like a pre-.NET incarnation of Visual Basic. I was able to overcome the initial learning curve, which included the crucial step of making mistakes and learning from them, then continued to simultaneously grow my understanding of the advanced capabilities of the language while reading about programming theory in general. Little of this actually happened while scrambling to meet deadlines...most of it took place after-hours, reading technical journals or in specific technical classes.

That's why it kills me when companies get upset about the poor quality of software, but are at the same time throwing poorly managed software projects at their developers and refusing to pay the cost of employee development. If corporations don't want to invest in sending their janitors to advanced skills training, that's one thing, but if they don't send their thinkers to learn the new tricks in an industry where things are always changing, they're only hurting themselves. Still, if you're not one of the lucky ones working at Computerworld's Top 100 Companies to Work for in IT, then you're often like me, forced to figure out this stuff on your own.

That's why staying a developer is tough. It takes work to improve yourself. There are ways you can get away without putting in the extra effort, and there are also things that can compromise your efforts even if you do make them. More on that in a future rant. But if you're serious about staying in IT, at some point you're going to have to learn something either because you want to or because the environment forces you to. But don't always assume the learning has to be in a new language. If you're using the right language (something that supports OO like C, PB, Java, C#, or VB .NET), you'll be able to make dramatic improvements in your ability by studying advanced techniques and applying them right where you are.

This is where chasing The Next Big Thing can be wrong. I think that getting over the newness of a language is a critical part of being able to move on to using it with advanced concepts. If you don't get to the point where the syntax becomes second nature, then you'll always be in cut-and-paste mode, the thing that might be the biggest cause of PB's demise. Cut-and-paste is a symptom of procedural logic, and procedural logic is the mental baby food of poor or mediocre developers. I can guarantee you there are still people slapping procedural logic routines into their Java and .NET objects and then patting themselves on the back and saying, "Look! I did OO programming!" No, that's "Uh oh programming." The people that never figured out OO while learning PB took their same bad habits to Java, then to .NET, all in their pursuit of The Next Big Thing. The Next Big Thing can't materialize because it's always being compromised by The Worst Old Thing.

Don't be fooled by the few in this industry that write the magazine editorials and work in ivory towers where they get to play with whatever they want and don't have deadlines. They write about how your programming tool is dead and you are crap if you don't move to The Next Big Thing. I say to them, "You're lucky, good for you, now shut the hell up, not all jobs are like yours. The rest of us don't want to hear about your time at the tech conference in Hawaii. We have to figure out how to resolve problems in a myriad of apps supporting a multibillion dollar company, and no, the app isn't written in the latest and greatest, but it's putting food on my table. Who's going to do that if I take a few years off to study an obscure API in .NET? You?" Staying a developer requires learning, but learning isn't always about a new language, sometimes it's about making better what you have, and it should be more about improving productivity than improving resumes.

Once you've achieved proficiency with a language, and are confident that you understand OO and have migrated from cut-and-paste to model-design-and-iterate, then you're probably at the point where you can start picking up all kinds of newer languages because at least some of the universally useful concepts are understood and can be translated over. Sometimes, if you know what you're doing, you can appreciate when the new thing, say Java or .NET, can offer you something your old language didn't like implementation of operational polymorphism through things like interfaces. But I'll bet a lot of the ex-PB guys that went to Java and .NET first heard about interfaces and said, "Huh?" And if you've used interfaces for a while, you know they are powerful, but like recursion, can get confusing if you don't design them well and apply them judiciously. Guess who probably implemented hundreds of interfaces without thinking first?

Anyone can become a developer. But if you want to stay one, try climbing to the next level before moving to the next language. Try supporting your own crappy code for a while, feel the pain those that support and use your apps feel, learn from your mistakes, and realize that there are better ways of doing what you think you're so good at. If you don't, you'll be dumping more of your crap on the ones that follow, you'll just be doing it in a different language. You can count on the lack of metrics in our industry to shield you for a while, but sooner or later, the smart ones are going to stop hiring you.

Recent Outsourcing Horror Visions

I'm hardly the first person to gripe about this, but there's an unsettling trend in America to send work offshore. The proponents of this practice defend it by saying that manufacturing and industrial jobs also went overseas but didn't compromise American supremacy.

Well, they're half right. There are some responses to this trend in the July 2005 Software Development magazine's feedback section (registration required). Michael Wheelen's letter succinctly identifies the entrapment of the American IT worker. An excerpt:

"We as American workers are stuck. Until cost and pricing stabilize around the globe, our economy will suffer the most because we have the highest cost of living."

And:

"The global economy is here to stay, but let's replace the old dogs with young dogs - people who understand the new technology and can manage it to everyone's benefit; young blood can create a win-win for corporate America and the public. For example, why not let the corporations outsource, but when the end product returns to America, fix the profit to a certain percentage of cost?"

Great letter, but of course what Wheelen is asking for isn't going to happen. In particular, with software, there's no tangible product to tie a profit to, and as IT veterans know, no software truly has a finite cost - as long as it's alive and being used and maintained, the true total cost grows with the life of the software. The old dogs have a little too much power too, and it always seems the young dogs that finally do replace them have learned their tricks from the ones that went before.

In the same issue, another letter, American Decline, from Thomas Ronayne serves a chilling image. He wrote:

"American companies are run by fools interested only in short-term results, and the offshore community is ready, eager and willing to
step in and take over. Because the relationship between government and industry in the United States is adversarial (compared with that in, say, Japan and Korea), we've seen the steel industry more or less vanish, the automobile industry get into deep trouble and the electronics industry disappear...we're on the rapid road to becoming a client of other countries that are willing to invest in their future. If we had to fight a war with Asia, we wouldn't be able to produce the basic materials we'd need to do so."

I hope that's a touch more sensationalist than reality would have. I'd read not long ago about how the steel industry in Philadelphia is making a bit of a comeback because after demand outpaced supply in other countries, the US was able to compensate at a competitive rate. Economic equilibrium is the key to making globalization less painful. Still, I can see the US being stupid enough to get to a point someday where the native industrial capacity could be completely gone, or enough diminished that if a wily agent of chaos could organize our allies to turn on us, there would be no foundation to support a future generation's "Rosie the Riveter."

But in the previous generations of offshoring, and in Ronayne's vision, the jobs being sent away from North American borders were largely physical jobs not involving soft skills. And if you look at the industrial offshoring efforts, not all of the work even left North American borders, moving to Mexico and Canada (ah, how arrogant we are to assume that America or North America always means "the USA").

But the know-how is still in the US! If there were a disaster, or a war with large scale scope, I'm optimistic that our engineers and surveyors and construction workers could find a way to adjust and still get the work we needed done at home to support a war effort. And North America is still a gifted geographical wonder, with natural resources galore, and still bordered by large bodies of water. You can compromise your opponent's industrial capacity and maybe even its economy, but no one gets a free pass from Mother Nature. Invading the US wouldn't be the cakewalk that Ronayne thinks it would be.

Mind Your Contractors

I don't like the way many corporations use contract labor, especially in the IT market. I worked as a contractor and as a full-time employee, and my experience found that most contract programmers are mediocre. There are of course some good ones. The outstanding ones have a thorough understanding of the chosen tools and best practices, sport a respectable work ethic, deliver quality results (including documentation, for all of you guys out there that were about to say, "Oh, I'm one of the good ones!"), and are truly professionals. But most don't deliver on all those categories, and they have no impetus to change because companies aren't rating their work properly. Instead, some mediocre contractors are getting by because they have a good network or have been friendly with some managers. They are not necessarily good programmers, but they're good networkers.

Where to use Contractors

I'm not sure if I just bashed or praised the mediocre contractors, but in a perfect world, here's how I see it being done. Corporations should use contractors for certain specific needs:

• To shore up temporary staffing needs on a given project
• To temporarily apply a specific set of rare skills to a given project
• To use an expert for either of the above reasons, but also as a mentor to the staff for learning new techniques/technologies

The problem I saw in the 90's was that contractors were being hired at times almost like full-time employees. They were given project management responsibilities and often a fair amount of power. That's not necessarily wrong, if the contractor is good and is the type of person willing to contibute energy to a turnover phase (teaching about and documenting their work for the full-time staffer that takes over). But many times I saw mediocre or bad contractors tasked with important tasks like system design and development. Giving bad contractors important work sets up the hiring company for lots of downstream pain.

A developer may be superficially proficient in a given tool, but still not be what you'd call an advanced developer. Some could even be certified in the tool, but not have an understanding of related necessary knowledge, have a terrible work ethic, or be a poor communicator. In an earlier post, I lamented that many developers simply haven't mastered advanced concepts and techniques like object orientation, database modeling, agile development, and relational theory. I'm not a PhD in those topics either, but I'm always pushing to learn more and I know enough to know the difference between good and bad code when I see it. Contractors in the 90's made, and some lucky ones today still make, rates in excess of $70/hr, and for the ones that don't bring to the table the advanced skills I'm talking about, that's about $40/hr too much. Many contractors fall into this category of guys that can put together a basic GUI but not really think long term about the user and how they're going to operate this screen and what they could do to make the user's life easier. And it gets worse if you look below the surface. The mediocre developer designs poor table structures and once you've got a bad database foundation, you've set yourself up for endless hassles making the GUI and business logic have to dance around that. The point here is that most developers fall into this category, and therefore by the law of averages, so do most contractors. It's not an intentional malicious thing; the market paid well and needed guys, and companies were hiring, so it happened.

The Metric System

Part of the problem is a lack of metrics in our industry. It's hard to rate the quality of someone's code unless you have the technical ability to read and understand it, and the advanced skill in that tool set to contrast the best practice solution with what the contractor devises. There's no way to rate a developer's past work without access to their code. So most of the time, the interview is based on personality match and a basic resume bullet match. Some interviews may include a technical interview too, but these are also flawed, often testing the interviewee's ability to simply memorize a tool's help file, rather than asking about how they would apply techniques to a solution; such interviews can identify blatant resume liars, but are often as much to boost to the interviewer's ego than to really test the prospective hire's aptitude for quality IT work.

Ironically, a contractor possessing a good work ethic and communication skills but flawed or dated programming practices is just as dangerous or maybe more dangerous than the stronger developer that might not be as comfortable chatting with the CIO. Guess which one is more capable of grabbing management's ear? Of getting the new project work? Of then creating more excrement in the company's systems?

Just Rewards (for the foolish)

Here are the risks you face when hiring a sub-par contractor and giving that person important development work:

• Most developers, contractors or not, have an aversion to documenting their work. When the contractor leaves, you'd better have resources skilled enough to pick up the pieces they've left behind.
• If the contractor was brought in to build a new system, they've gained valuable business knowledge (or should have) as part of the analysis process. When they walk, so does that knowledge...often to a competitor who's hiring contractors to build a system and wants someone to have business knowledge on the resume. Also, the full-timer tasked with picking up the pieces has to go through the learning again because of the contractor's likely reluctance to document.
• If the contractor didn't employ good OO design, then the code will likely be full of hardcoding and old-fashioned procedural-style logic. That might have been ok in 1990, but by 2005, these guys should know better. Polymorphism is a very powerful tool in the hands of the right developer, and it is a good solution for addressing the many custom conditions modern systems need to handle, and the business rule changes certain to follow. Systems written by guys uninterested in educating themselves on newer techniques (techniques, not necessarily languages) will be hard to maintain and prone to heavy production support, which is a poor time sink for your full-timers left holding the bag.

"Let's build a crappy system!"

There are good contractors out there. As a former contractor myself, I strived to not produce the deficiencies I mention above. I always documented my work, often with formal manuals that could be passed on to those continuing my work. And I worked to apply best practices and think of the user when designing things. I made my share of mistakes, but by and large my clients were pleased. I enjoyed working as a contractor and getting to see a variety of projects and work with different people, and I might still be doing it if the market hadn't crashed. But as a full-timer now, I'm dealing with the things I mention above. The companies that hired the contractors weren't purposefully saying, "Let's build a crappy system!" but they had no way of knowing what was going on because they didn't have the ability to evaluate the true quality of the work.

Corporate America vs. The American Dream

It's sad to think about how the relationship between corporation and employee has changed in America. At one time, you rarely saw people switch jobs; they took care of their employer and the employer took care of them. There was a sense of duty and honor on both sides. Things are different in the present. It's common to see people switch jobs with great regularity.

What changed, and what caused such change? That's a question with perhaps many different answers. Corporate America will say employees are selfish and greedy and job hop to secure a better financial position. Workers will say that corporate America started it first, by laying off droves of employees while corporate heads got rich. Like him or not, if you've been in corporate America for any length of time since the 80's, you've got to have some appreciation for Michael Moore's sentiments when he refers to Roger Smith's move to offshoring in Roger and Me, sarcastically saying, "Roger Smith was a genius." I say to the pointy haired ones, "Yes, employees job hop to secure a better position, but that's better than doing it by selling your soul."

I'm not sure what side of the chicken and egg question comes out ahead in that argument. But I do know that the rich have been getting richer and the poor getting poorer for a long time. And it's a trend that needs to change before there is no middle class. I'm not an economics expert but it seems to me the middle class is the foundation for nearly every society we've seen. George Carlin joked that there are three classes: The rich make all the money, the middle class does all the work, and the poor scare the middle class into keeping its jobs. Carlin can be goofy at times, but in this case, he's right.

Do all the work! Is that all the middle class does? I said in an earlier post that you can't really be a good leader without some decent followers. Ever worked on a team where there were lots of people who thought they were king? I've seen it and it's not pretty. Too many chiefs and not enough indians, goes the saying, although perhaps the more politically correct version of that should say, "Too many Ghandis and not enough Patels." Damn it, someone has to do the work! We're a long way from the day when we have to worry about robots being able to make the human race obsolete, so that means there better be a middle class around to get things done. And when you think of America, after all, isn't the "work ethic" one of things that always comes to mind?

What would a world without a middle class look like? It depends on where you are. If you're one of the rich, it looks pretty damn good. You don't have to work and the masses will do anything for an opportunity to eat the crumbs off your seat cushion. But if the distribution stays the same, then it will mean that the rich will be outnumbered 9 to 1. Of course, the politicians will be on the side of the rich, so the poor won't be able to count on tanks and soldiers to support their thoughts of revolution.

What's the point? Just that perhaps for America to protect itself, it may have to alter the concept of the American Dream. That's an arrogant term anyway, since to improve and advance one's self isn't a goal indigenous to American soil. It should be termed the Human Dream, or perhaps the Human Aspiration. America was just fortunate it was in the right place at the right time, with the right people, and um, a powerful military, to make things happen. It became the geographical embodiment of this Human Aspiration. The land of opportunity.

But the world is changing. As more of the countries in what Dr. Thomas Barnett calls "the non-integrating gap" begin to come to grips with their disorder and civil wars, as more countries like India produce educated lower-cost alternatives to American labor, soon the "American Dream" will begin migrating to other parts of the world. Where before wars and political instability made countries infertile ground for the Dream, now there are more places where it can happen. The laws of supply and demand dictate that naturally the world's best and brightest won't need to make the pilgrimage to North America to make their lives better.

To combat this, perhaps America must make the transition from a country of great expectations to a country of great managed expectations. And this brings me back to corporate America, but also to the population as a whole. Our future leaders are going to have to convince people that exponential company growth and profits in an age of subdued American supremacy aren't realistic. They'll have to show that the Japanese have it right when the difference between the lowest worker's salary and higest executive's salary is a factor of 7, not 70 (and that number is even higher now in America - I'm quoting numbers from the 1980's). No human being is worth that much money. And not all the rich end up being philanthropists like Paul Allen, funding advances in aviation and spaceflight technology, or Bill Gates, funding biomedical research. Excessive money can sometimes be like excessive time; it finds not-so-nice places to trickle into, like politician wallets.

We need to learn that perhaps it's unrealistic to expect such gross gains from our stocks and even in our personal lifestyles. I know it sounds horribly un-American here, but how expensive would medical care costs be if doctors decided a Toyota Camry was good enough for them instead of a Mercedes? Maybe that's a bad example. But if everyone had more modest expectations (not that they gave up on improvement, but learned to not sacrifice all for strictly monetary gain) then perhaps you wouldn't see such greedy moves on the part of corporation heads. And perhaps their usual excuse, that they're doing it for stockholders, would stop holding water.

A Great Cautionary Tale

I don't know why I didn't think of this the first time I saw Lord of the Rings: Return of the King, but seeing it again reminded me of a cautionary lesson everyone needs to know. Not just IT, but America as a whole. There's this wonderful scene at the end where Aragorn as the king sees the Hobbits pay their respects to him by bowing. He tells them, "My friends, you bow to no one," and everyone there, the king, the soldiers, the heroes of the dwarves and the elves, all bow to the Hobbits.

If I'm remembering correctly, this was already explained in professional criticism of Tolkien's work as his warning that mighty heads of state should not forget the little people that make up their constituency. This is hardly a ground-breaking moral. Leaders have been compromising their oaths for personal gain since, well, probably since Tomak the caveman leader first clubbed someone else's wife over the head because he was tired of his own. As humans, we've proven time and again that we just can't control ourselves. We've really got to work on that.

But without the people doing a leader's bidding, we would be nowhere. I'd give our societies less than a week to dissolve into chaos if the little people in charge of getting water to our homes decided to quit. Or less, if they also cut off sewage operation.

The moral is simple for world leaders: You cannot be a leader without reliable followers. It's also obvious for businessmen. There's always much talk about how to be a good manager but if no one follows, even a good manager is useless. Any manager can look good if he has great resources. But the greatest managers do it facing compromised resources and unexpected challenges.

The Pretension of Profession

Another cautionary entry, but not entirely about IT. Infoworld's Chad Dickerson again writes a good one in his May 9, 2005 column. He's talking about how people are freaking out because of the heavy loss in interest students are showing in the computer science disciplines in college. But he makes a great point about it at the end, saying that the lack of a computer science degree doesn't mean you can't be successful in the IT world.

This is too true. We, and not just IT workers, but everyone, can easily succumb to the ego trap of assuming our job is important and special. We do this because we're humans and we want to feel important and special, even if that means treating others like rat piss. Don't lie, we've all done it at one time or another, even if we weren't intending to hurt or look down on someone else. And one of the more obvious ways we do it is by asking the innocent-sounding question, "Where did you go to school?"

How do you respond to the answer to that question? If the person says they went to the same school you did, you think, "Oh, this person is OK, they're like me," and much feigned camaraderie and back slapping ensues. If they mention some prestigious school, perhaps the response is one of jealousy, or respect, or of feeling intimidated. If they reveal a less prodigious institution or perhaps the fact that they didn't go to school, well, now you got 'em, don't you?

I know, not everyone falls to such base behavior. But Dickerson's article reminded me of the pretension we can all take with our professions, our degrees, and other nonsense. Sometimes it's good to stop and remember that we're all just little pieces of excrement in the universe. It's not right to discount others based on titles. Don't assume that just because someone didn't study computer science they can't do computing (conversely, don't assume that someone that studied computer science is always a good developer).

Here's a little secret: anyone can be a programmer.

All it takes to be a decent programmer is the capacity for logic, basic mathematical ability, and the ability to communicate. When I was at EDS, one of the things I was impressed by was that this company, known for being a huge computer firm, hired people from all walks of life. I worked with English and geography majors that ended up being good, hard-working, competent developers. It was more about a sense of professionalism than simply having been brought up in a computing-related academic discipline. EDS says that this was all part of the plan to build diversity in the company, but that computer novices would take lower salaries probably wasn't lost on management (sort of like how cruise lines brag, "We have an international staff" when the reality is that they have to because only people from third-world countries would toil on those ships for a pittance).

I think one of the nicest things I ever said was when a lady and her son were talking to me about schools. He was a good kid but struggled a bit academically, and was going to go to a junior college. She asked me, "Is it really that big of a difference, to go to a big name school?" I could sense that she'd probably been chatting with "friends" and when the hens all started clucking about the schools their kids were going to after graduation, she got to see the undersides of their shoe soles.

The truth of course, is that despite my diatribe here against pretentiousness, going to a big name school does make a difference in that first job hunt. Not because the big name school was necessarily better, but because people are pompous, and because we all have to live by rules we didn't write. But I told her a different truth, saying, "An education is what you make of it. It doesn't matter where you go. It's about how hard you work and what you want to learn." And that is the truth, because I went to a fairly respectable university, but I know guys that went to JCs and got, in some cases, better instruction than I did.

Software development: Art or Science?

I'm going to try and set the foundation for future discussions here by thinking about some issues that I think are really holding IT back from advancement. Metrics, developer evaluation, management, development techniques, and outsourcing are all subjects to come. Right now I want to discuss something that's bothered me for a long time.

Do we know what the heck software is?

Of course, you say, we do, it's that collection of zeroes and ones that power an entire industry in the technology field. But with regard to development, why is it we keep shooting ourselves in the foot in IT projects? The answers can come from a dozen different places. Management is always a major factor, but so is the lack of metrics in our industry, and that the technology is still relatively new. That it's always changing doesn't help.

One of the problems is I think a misconception about it. Software and hardware both come from the discipline of computer science. Well, that word, science, implies reliable predictability. If you're a developer of any experience, you know that users, software, languages, hardware, and business requirements are neither reliable nor predictable.

Thus the struggle I have with the question, "Is software development an art or a science?"

It's a science because we've seen how the hardware we run on depends on the laws of physics and we're beholden to those laws. When we code, we know our code follows certain logical pathways and we know that features and abilities in our languages will uphold (most of the time) repeatable results, and that we can usually identify a change and its effects on our code. That sounds pretty scientific.

But it's an art too, isn't it? We start with a blank screen and with work, we end with a page of script or code that can do some amazing things. Software developers don't just crunch numbers, they also design graphic user interfaces that communicate with humans, digital interfaces to communicate with other machines and operating systems, and are continually striving to modify the code for processing efficiencies. Those things all require some creativity. Designing database schemas with efficient and maintainable structure, that's a task involving both science and art. Designing a software framework that can reduce maintenance hassles down the road, that takes real ability to plan and to process and research past experiences. It's not just about hammering out a couple IF statements. And when you look at it that way, it can sound pretty artistic.

And I think that's one of the problems in IT. Nobody really understands software. HR people, managers, and executives think IT is just a cost center that wastes money on that thing called applications development. It can't be that hard, nice software like Quicken only costs $30 after rebates, right? NO! Not right. Slapping out some code is easy, but making it sing is hard. Making it sing in tune is even harder. Making it sing in tune, in a choir with other apps like Oracle Applications, SAP, and Windows, across an unreliable phone line to a remote office...it isn't easy, and some of those expensive choir members don't sing too well themselves!

But people at the top are often too quick to assume that computing technology is all simple science. Few realize that it's much more variable and complex and contains unpredictable artistic elements. That is, I suspect, one of the leading causes of misguided project plans and therefore a cause of having to burn the candle at both ends.

Here's another problem: managers and executives aren't the only ones that misunderstand the complexity of software. You know who else doesn't get it? Software developers! Yes, that's right. In my short 15 years in this industry I've figured out that most of us are complete morons. Mediocrity is threatening the quality of applications companies use, and that's debatably just as damaging to this industry as any mediocre manager. When I was contracting, I couldn't believe how many of the contractors were complete idiots. They could put together some basic things, but good grief, when an entire population of them didn't understand or care about database theory and didn't strive to perfect their craft by studying advanced techniques, it just doesn't speak well of us as a whole. There were and are guys I've crossed paths with that completely dismiss the artistic side of the development equation. And of couse those guys are getting paid more than me to work half as much, but that's a discussion for another day.

Anyone can figure out the IF statement. But it takes a real professional, one who equally reveres the scientific and artistic sides of our business, to truly earn the title of software developer. That means doing the artistic parts that aren't so fun: interacting with users to find out what they really want, learning the business, striving for code perfection in both function and form, DOCUMENTING their work for both other developers and the user community, and remembering always to never stop learning.

H1B Visa: Threat or Benefit to America?

I'm not going to get too deep into this because there has already been plenty of other discussion and most of the IT guys out there already have an opinion that I'm not likely to change. But I've worked with several H1B employees in the last five years, and I'll share my thoughts.

The corporate line is that they helped fill a gap in labor skills. This is probably somewhat true from an IT perspective. Corporate America doesn't like to spend money to develop employees internally. To a ruthless businessman, that makes some sense. But a smart businessman, a patriot, an intellectual, an economist, or a sociologist might have a valid argument if they said that philosophy was short-sighted. I could fill fifty more blogs on that topic, so I'll get back to my H1B experiences.

I didn't see so many of them in the early 1990's when I was a full-timer in the EDS army. But as we moved to the mid-1990's we did pick up a few contractors and some were H1Bs here under the mantle of some consulting firm. When I broke away and went contracting myself in 1997-1997, I started seeing a lot more. By 1999, it seemed like 40% of every IT department I worked with was Indian!

In my 1998-2000 stint with PG&E (nutty company, but boy the folks there do take good care of their people and I will always respect them for that), my office mate and I, both frequent HBO watchers, saw a parallel between HBO's OZ and our IT department's diversity. OZ featured several factions, often aligned by race: the Hispanics, the Blacks, the Italians, and a few straggling misfits. My office mate would say our group was like that: we had the Chinese, the Vietnamese, the Caucasians, the Indians, the Mexicans, and the Visual Basic guys.

Anyway, I digress. Was this proliferation of H1B workers good or bad? If you were an unemployed IT worker in the last five years or so, you'd understandably want to say it was bad. But let's take a closer look at the situation.

First of all, was it really cheaper for Corporate America to hire the H1B workers? I used to think so, but now I'm not so sure. I spoke with many of the H1B holders, and they told me that they were being screwed by their middlemen just as American contractors could be if working for an unscrupulous company. Several of the H1B workers I knew were getting paid annual salaries of 60 to 80K. That's a darn nice salary, particularly if you were sending a chunk of it back to Bombay every month! Knowing what we know about sleazy middlemen companies, they're not going to put their employees to work without a profit, so they were charging the American company perhaps $50-$80/hr in contract rates. Folks, that might be a little lower than what companies had to pay for an American contractor in the late 1990's (perhaps between $70-$90/hr with a those lucky SAP bastards making over $100/hr) but I wouldn't call it cheap. $50/hr is still going to be higher than the base salary of most full-time IT workers.

It seemed that many of the H1B workers were getting screwed by the middleman company. They might have gotten some modicum of benefits and tax coverage, but I know how these "consulting" firms work: they like to bill you out at double what they're paying you. I know, because I got royally screwed by the American counterparts of the Indian firms when I was contracting. There's a future rant to be had on the whole contracting business. But we're talking about H1B workers, and I really don't think they were that much cheaper for Corporate America, based on my experiences. I theorize that the issue is not only cheapness, but also impatience. Corporate America didn't want to wait for employees to be trained, it wanted to get going right away.

Oh, there are economic benefits, to be sure. Corporations liked the idea that the H1B guys were contractors, didn't need benefits, and could be released at will. Because there were so many available at the time, it was a natural matter of statistics that many H1B workers would pick up the contract work. But where I worked, lots of American contractors got work too. I really didn't see this as primarily an issue of economics, but again, that's just my experience.

Not that there aren't economic affects of the heavy H1B availability. The simple laws of supply and demand state that with so many of tech workers available, the rates they'd have to be paid would drop. And clearly in that regard having a lot of H1Bs around didn't help the indigenous population of IT developers, especially contractors. But I just don't see the pointy-haired ones as having planned this fallout, as so many disgruntled IT workers claim on bulletin boards and blog comments. Some of the corporate heads are soulless and selfish enough, certainly, but I'm just not convinced such a machination was nationally coordinated and executed by them and with such economic forethought.

I'd like to see the American worker get that job, and still see a steady but regulated stream of immigrant workers. But if we had to choose beween using H1B workers or outsourcing, I'd pick H1B. The H1B worker may be sending money out of the country, but as long as it's not for terrorism, then that's his or her right. What I like in the H1B option is what's happening to that other half of their paycheck. They're getting taxes withheld just like everyone else, but they're not going to be claiming any of that money unless they become citizens, and many did not have that goal (is it fair that they would pay taxes for benefits they can't claim? Yes, because they ARE claiming some of the benefits the government provides, like law enforcement, transporation facilities, and other infrastructure services, not to mention insignificant details like THE JOB). They also had to pay rent, buy gas, food, and pay travel fees. With outsourcing, all of that goes away. In this regard, the H1B program could be called a benefit for America, or more accurately perhaps, the lesser of two evils. It's certainly a labor practice that has more benefits than total outsourcing.

In any event, I'm not against immigration, because if you truly love and know America you know that immigration is what America is all about. Everybody here came from somewhere else, except the Native Americans, and who knows for certain that they didn't hop over from some other geographical plate during an ancient earthquake?

The problem isn't H1B workers or immigration. Those things are what help keep the world's best and the brightest coming to America (though the H1B limits can probably be adjusted to encourage Corporate America to develop existing employees). The problem is a self-absorbed and impatient America that thinks in terms of quarterly profit reports and not about the long term effects of treating human beings like cat shit.

You know who treats their people well? The Indians. How did they suddenly show up in the 90's with all these technical skills? They have a strong emphasis on education, and I found out from some of them that an education there is very affordable. In America, taking a class in a new language might be $3000, a bit high for a salaried IT person responsible for a family of four. But in India, you can take that class and more for the price of a lunch. And they can get cheap versions of our technical books too for about a buck. These books are legal...made by the same publishers that sell us the exact same text on slightly nicer paper for $55.

I've got nothing against the Indians that are taking advantage of what's been put in front of them. But if our IT pundits, company heads, politicians, and universities are going to complain about the lack of skilled American IT workers, maybe they should also consider leveling the playing field. That means addressing the rising cost of tuitions, investing in employees, analyzing H1B limits, and reconsidering the tax breaks and protections offered to companies that outsource.

Oops, I wasn't supposed to go too deep on this...and come to think of it, I didn't...I've only barely scratched the surface.

Expert Opinion

Well well, I barely get my first major rant out the door and there's a couple of articles in this week's weekly IT magazines talking about related issues.

Paul Glen writes To Motivate, Don't Demotivate in Computerworld (2 May 2005). Pay attention to the part where Glen refers to "excessive monitoring" and relate that back to the last The Burning Ends entry. Also note his mention of internal versus external motivation. Great job, Mr. Glen, and well-timed to keep my last rant from making me look like a complete idiot.

Chad Dickerson of Infoworld asks Does IT demand too much commitment? in his weekly column (2 May 2005). I think IT does indeed ask a lot of people, but there are different inflections of overtime:

• Forced, to meet a deadline
• Forced, not for deadlines, but just because someone (a jerk) thinks it will improve productivity
• Forced, because of an emergency. IT guys do this all the time when a system breaks at odd hours of the night
• Self-imposed, where a dedicated worker does it either because he's trying to make things happen faster, or he's just crazy

More discussion on that in a future rant.

I like both of these magazines. Neither really has any deep technical information like the stuff I used to get from Sys Con Publications (like their various Developer's Journal series for Java, PowerBuilder, .Net, etc.) but they do a decent job of skimming the headlines for things pertinent to IT folks. Strangely enough, I don't always read every news or tech piece they print...my favorite section, the one that always seems the most useful to me, is the management section in Computerworld.

I try to keep up with these magazines, along with other freebies like Software Development, and they'll make for some good conversation points in future blogs.

Cautionary tale: The Burning Ends

I call this thing a blog that's mostly supposed to be rants about lessons I've learned in IT, so here is the first one. And it shouldn't be a surprise that it's about management (or lack thereof).

Managers: I know you think you're big shots because you carry the title of "manager" but really, it doesn't mean you have carte blanche to be an inconsiderate buffoon. Most of the IT guys I know are dedicated enough workers that they don't really need a manager to provide a lot of "rah rah" motivation or whip cracking. We love what we do, we like the problem solving elements inherent in applying technology or developing software, and the work is largely its own reward. So when times arise that sometimes require "burning the candle at both ends," developers aren't shy about answering the call. But the way the human mind works, motivation can be internally driven or externally driven, and sometimes external "motivation" by a pointy-haired manager can end up doing more damage than good. I have a half dozen personal experiences to illustrate this, but I'll refine this diatribe to include only the two best ones.

I used to work for a huge data processing firm...those of you that know me already know the name of that firm, but to protect the innocent, I'll just call it something like, oh, Electronic Data Systems. This juggernaut is the classic example of a monolithic company often called a dinosaur by business analysts. Well, I still own stock in this place, so I hope they can turn it around, but the reality is that when you're 75,000 people large (circa 1995), you're bound to find both good apples and bad apples. And I had the fortune of knowing several of the not-so-good ones.

First Tale: The Theory X Manager

This manager was the worst I've ever worked for. I would be unlikely to accept working for him again even if I was starving and he offered the last job on earth. He believed in surveillance and keeping employees under his thumb. They were not resources to be cherished and nutured, they were to be exploited. He was a real gem of a human being too, and I'd heard of several unprofessional acts in the history of his time managing this site (he was dating one of the secretaries - son of a bitch couldn't even pick a better cliche). In management school I learned of William Ouchi's Theory Z (The Theory Z manager manages projects, yes, but also develops people. They're cognizant of the fact that their subordinates are human beings and might like things like a decent work place and that a happy employee is more productive than a fearful or forced one). This person was just the opposite, a total Type X guy. He'd sometimes berate you based on hearsay from other employees, rather than discuss things direcly with you or your direct report first to get the full story. And he was a cheap bastard and didn't spend a dime on improving conditions for the staff. No training, no new technology, nothing. And IT workers passionate about their work typically do like to learn new things. We are like any other employee, we like pay raises too. In the nearly two years I was there, there wasn't a single monetary adjustment to my salary, and I'm betting the other employees didn't see much either. I'd heard rumors this cheapness was because EDS managers got bonuses for staying under budget and so had little impetus to spend on the staff. I don't have proof that was the case, and I'm certain the perpetrators of such a policy would deny it if asked. But damn that story fits pretty well and would certainly explain the selfishness I witnessed.

I worked hard at first, but by the time I'd finished my assignment there, I was bailing out at the 8 hour mark at every opportunity. I couldn't stand being there. I knew excellence wouldn't be rewarded or appreciated, so I didn't feel I should waste excess time striving for it. I was admittedly less mature then and also less skilled, but I still contributed and when you look at my total career, there's no question about my self-motivation. Few work a longer day on average than I do. But this guy managed to crush it. If he saw the quality of my work now and the amount of work I put in, he'd probably act surprised and ask, "Why couldn't you have done that for me?" BECAUSE YOU'RE A SELFISH ASS WIPE OF A MUSSOLINI WANNA-BE, OKAY? If someone was stealing from you or mistreating you, would you spend a second more around them than necessary? How about working lots of unpaid overtime for them?

Look, developers are pretty good people on the whole. I know there are some that aren't that bright and shouldn't be in this industry (thanks a lot, 1999!), and there are others that aren't as motivated as you'd like, and others that could use a major boost in the people skills department. Sure, I've met a few that were prima donnas, but every industry has those. And there are some criminals in the IT ranks (data thieves, pirates, inaccurate hour billers, and salesmen). But typically, IT folks are intelligent and reasonable enough as a group that they're no more high-maintenance than any other group, and quite possibly a lot less. All you have to do is show some respect and keep them fed with decent projects and some doughnuts. They'll reward you with quiet loyalty. For that matter, you could probably apply that management point to all employees.

Second Tale: The "You're just a Robot" Manager

I'm a new guy on a new development project that's pretty intensive. This is a big project that's building a system for the company to branch into a new segment of business. It will ultimately fail because of the oddest combination of quirky personalities, mismatched skill sets, and lack of experience that I've ever seen in my career. The manager I first start working for is not a bad person. He's nowhere near the Theory X guy in the first tale above. But he was clearly inexperienced or ignorant about the people side of management and was totally clueless about how to motivate people. Perhaps this is not totally his fault; he was a technical guy promoted to management, probably without the managerial training and social evaluation that should accompany such promotions. I don't know if this was a Dilbert Principle deal - I think he might have been a decent technical person in the right niche. Anyway, we're running behind on the project and people are already working hellacious hours (12-16 hours/day) mostly by virtue of self-initiative and the desire to be professionals. This is the truth and I can find witnesses if you don't believe me. We were coming in at 8 every morning and some of us were staying until 11pm (yes, most of us were single). Can you sense what's about to happen? He's going to mess it all up.

He and I are discussing the project schedule for the next few weeks before a pending deadline. There are more hours on the recently reworked estimate than there are 8 hour days before the deadline. "Oh!" he says, getting visibly excited, "If you put in some evenings and weekends, why, we'll be just fine." My heart sank. I had already been putting in extra time every day, including Saturdays and Sundays, but it was by choice. It was, foolishly, perceived by me as my duty and my honor. I wasn't thrilled about it, but I did it and didn't need to be asked. The minute it became forced, the minute the manager changed the focus from self-initiative to external pressure, the internal motivation withered. This manager actively and openly disregarded my personal time by saying what he said. I gave the time (which we were doing anyway), and what do I have to show for it now? Nothing. Two years between pay raises. No bonuses, no vaseline. The kicker? This manager left, saying, "My team doesn't like me because they want me to work overtime like them but I can't, I have a family." I can't put my response to that in printable words and still retain some professionalism.

Respect your people, and by extension, their time. Don't just assume they have nothing else to do during their evenings and weekends. Don't plan projects by pulling delivery dates out of your ass. Don't expect miracles if you can't acquire the best training and tools for your team, and rememeber the mantra that I'm going to spout on this blog until I die: Making software isn't hard, but making it right is. That means quality people are worthy of a quality salary. There might be a day when all developers truly understand OO and best coding practices, and the general quality of code is truly a mere commodity (fat chance, but you can believe it if you want). But the labor issue doesn't change. You're not just paying for skills (which change every ten years anyway), you're paying for intelligence, loyalty, and dedication (things you need all the time), and those should be expensive things, regardless of what continent you search for them on. I know, you're remembering the crazy 90's when you had good folks leave for better opportunities elsewhere, but if you were paying 1990's salaries insted of 1980's salaries, that might not have happened. There are good ones out there...you're probably not hearing about them because they don't make a lot of noise or move around much. They're dutifully working.

If you're a manager learning that from me now, then that's pretty sad. If you're a manager and you're shaking your head and not understanding my point, then you SHOULD NOT BE A MANAGER because it's likely there are people suffering under your command.

Blog vs. article

I can see why blogs are popular. For the same reason that the Internet in general became popular. Any idiot with a keyboard and a delusion of literacy can suddenly become "published." When the masses put their weight behind the online printing press, they enable an incredible amount of rapid communication but the oft-mentioned signal-to-noise ratio becomes an issue (not that there isn't noise in the print world too).

As a professional freelance writer (albeit part-time - it's like acting or making music, I suppose, only a few of those doing it become rich) I thought the blog might be a nice place to share thoughts quickly, and it is. But I struggle as I begin my blog with the classical nature of the article. In most of my published work, I have to research the topic, make efforts to get facts right, and know too that an editor will see the work before the public does. The concept of a blog seems kind of sloppy to me. It doesn't have to be, and I've seen several impressive technical blogs laudable for their detail, some even including code snippets, graphics, and thorough analysis. The blog of Raymond Chen comes to mind, but I'm not a .Net bigot, it's just an example of what I consider a blog of professional caliber.

I wonder, how much time are people spending writing these blogs? Some of the nicer entries are obviously researched and well presented. Do their writers have a considerably more flexible time schedule or are they simply of remarkable and superior talent?

One of the masses now

Google's freaking amazing. They provide all this stuff for free? How do they do it? Ah well, I've got a blog now. I'm one of the masses. I'll have to think of something else to say.