Tuesday, May 15, 2018

Used Car Buyer's Guide

I hate buying cars. New, used, it doesn't matter. I've mentioned before what a miserable environment the car sales trade is and not much has changed except that the Internet can help keep you more informed.

So I tend to drive my cars for a long time. The good news is that I don't have to deal with car salespeople very often. The bad news is that I don't get much practice dealing with car salespeople. But I recently had to shop for a used car for my teen who is now driving and wanted to summarize my lessons learned for future reference. If it helps you too, great.

One other thing: language alert. I've said it before, I don't know how a normal human being can discuss the auto industry without a few expletives.

Accept that You are Fighting the Weight of an Old Industry

The auto industry has been around a long time. Longer than anyone reading this has been alive. And I don't know when it became such a contentious practice to buy a car. Maybe it's been full of bullshit since the beginning, I don't know. But today, the best you can do is say, "It is what it is," and then do what you can to be informed.

First Steps

Understand What You Want

You could just browse any used car inventory until you find something that makes sense to you, but really you'll be much more efficient by simply listing and prioritizing a few items that are important to you. With any car, features like fuel efficiency, styling, space, comfort, technology, and safety are typical concerns. If you're looking for used, these are additional items to consider:
  • Condition - Understand that a used car is likely to have picked up a few scratches even if it's still relatively young
  • Miles - The mileage the vehicle has collected is a part of the condition, but is also important in evaluating whether there is any manufacturer warranty left on it
  • Age - Years old...just like you use it to judge humans 

Use that Internet

The Internet is your best friend when it comes to auto buying. It isn't perfect; sometimes it's a frenemy because the auto industry also uses it and the Internet can't provide full transparency over a shady industry, but it beats the heck out of nothing. And it's a lot more than nothing! You can get expert opinions on any car model, stats on everything from fuel ratings to vehicle weights and tire sizes.

Find What you Want

When you have decided on what model you would like, the Internet can also help you research what's available in the market. The problem is that there is no one perfect site that can solve all your problems. Not all sites access the same inventory so you'll be hunting across several of them. Here's a list of what I've used along with some comments:
  • TrueCar: I wrote about TrueCar before when I bought a new car. It's handy, but has changed a lot over the years and isn't quite as good as it used to be. However, it's still one aggregator that can help you find cars at traditional dealerships. TrueCar also lists a price, but don't take it at face value, in most cases you can still negotiate. TrueCar search results seem to include only larger institutional dealers.
  • CarGurus: This is a relatively new site I think, and I kind of like what it had to offer. CarGuru aggregates available listings but it also adds some useful information. It claims it studies listed prices for that model and will give you an indication of whether the deal is good, fair, or overpriced based on what it sees. It also has handy information for negotiations, like how long that car has been listed. Generally speaking, dealerships are not interested in used inventory hanging around for more than a few months. One salesman told me that with their used inventory the prices will gradually drop every few weeks if it isn't sold. I've actually seen this as I was hunting for my latest used car. 
    Unfortunately for Texas residents, CarGurus recently stopped including all the price history supposedly due to government regulations. That sucks, although not too badly; it basically was a way to track the dealer's changes in price. So on the car I ultimately bought, you'd have see it originally listed for around $12K, then $11,800, then $11,680, which is when I went to take a look at it. However, the data on how long that car has been available is also useful and gives you an idea of how serious a dealer might be to get rid of the car.
  • AutoTrader: Another big aggregator. Easy to use, searchable, and sortable, but otherwise unspectacular. I have noticed this site will include independent dealers along with major ones, and also a few private sellers.
  • Cars: Another aggregator similar to AutoTrader. Features lots of great searching and filtering options. Results seem to only include dealers.
  • Craigslist: You've heard of this site...it's the site where you can buy almost anything direct from a person. The world's biggest garage sale. This is where you want to go if you want to buy direct from a previous owner. But there's some risk; you don't know anything about the car and since there isn't a partnership between Carfax and Craigslist, there isn't a free Carfax link for the vehicle. This site is a throwback to the wild west days of the Internet, back before the days of eBay buyer and seller protection. It'll be up to you to judge the seller when you meet him or her. You can probably find a great deal here and avoid a dealer's dirty tricks, but you could also end up with a car that has problems. For all the crap I give to dealers, at least they generally favor better used cars for their inventory. Guys in the car industry I've spoken with say the total beaters (the worn older cars) from trade-ins are usually sent to auction. 
  • Ebay: The famous auction site also has cars in a sub site called Ebay Motors. It includes listings from dealers and individuals. It distinguishes itself from other sites by including feedback about buyers and sellers if they have any history with eBay. Buyers also get some purchase protection against fraud if they complete the transaction on eBay. This might actually be as good as what you might get from some dealers, and likely better than what you'd get from a sleazy independent dealer. Texas Direct Auto lists lots of cars on eBay. 
  • Edmunds: Another aggregator that like CarGurus will also try to rate the quality of the deal. I found this site's performance janky.
  • Kelley Blue Book: Since 1926 the Kelley Blue Book has been a resource for vehicle valuation. One of the funniest things that ever happened to me when buying a car was when a salesman asked me where I came up with a price when I made an offer on a car. I mentioned KBB and he acted like he'd never heard of it. It's only been around for a hundred fucking years, dipshit! According to the FAQ on the KBB site, KBB was bought by AutoTrader. So I found it interesting that their sites look so different. KBB, like Edmunds and CarGurus, is a listing aggregator and will try to rate a deal's price. The site also has a valuation feature where you can check what it thinks the value of your car is. One thing I don't like is that KBB's site will detect if you have an ad blocker on and ask you to turn it off. 
  • CarMax: I mentioned CarMax before in a previous post. They tend to feature good condition used cars and have no-haggle pricing that is ok although you can probably do better elsewhere if you're a good negotiator. They are also one of the better places to sell your car. I noticed their listings do not appear on all the other sites so if you want to review their inventory you probably have to go to their site. 
Those are just a few of the sites I have used. There are more, I'm sure, and more to come. The Internet has made it possible to see cars from everywhere although of course you can filter the results to listings closer to your location.

One thing I want to say about the sites that try to rate the quality of a deal is that you can't assume that rating is always correct. Like a lot of high-level statistics, it's sometimes a handy tool when scanning dozens of listings. But a lot of those listings could be off. One of the cars I looked at had an Internet price of $13K, but a sticker of $15K. Depending on what price the dealer put on the listing, there could be some wild swings in the evaluation of the deal. And especially with a used car the price means NOTHING until you get a chance to inspect the car yourself.

A Word about Carfax

Most of the listing aggregator sites mentioned above, and most dealer sites too, will include a link to Carfax so you can see the Carfax vehicle history. This can be useful, but understand that it can also be flawed. Carfax can only show what is logged in the databases that it searches. At best, you'll see everything from when the car originally rolled onto a dealer lot, to every time it was registered with a DMV, every accident it was in, every time it had an oil change, and every time it was put up for sale.

But in my experience not all of that data will be there. Not all maintenance shops record oil changes. It's also possible for a car to have been in a minor accident that wasn't reported. I know this firsthand because I've seen it twice. When I bought a 2007 Nissan Altima from Texas Direct Auto years ago, it had dropped in price quite a bit and I was happy to have it but after living with it for a while I realized it had been in a minor accident. This didn't seem to affect the engine and the car was pretty solid for me mechanically while I owned it, so I don't feel too badly about it. But the Carfax was clean.

You Never Know

The Carfax shortcomings point to the biggest unknown in used cars. You never know who had it before you. That's the biggest mystery. Was it a nutty teen driver that drove everywhere fast and had several cosmetic scrapes (or worse!) that have since been patched? Was it the cliché about the old grandma that only drove the car gently and on Sundays? Carfax can tell you if it was a former rental, so that helps some but also opens up even more mysteries: how many assholes drove that car when it was a rental? I don't trust former rentals; there's no guarantee that a car with only one owner was well maintained, but a rental is likely driven by dozens of people before it went on sale, and this increases the risk that at least some of the previous drivers didn't take care of the vehicle.

Carfax can tell you something else you may want to consider. Cars from the areas with harsh winters are likely to have been driven on roads where they are exposed to salt. Salt is not a good thing for a car, so look on the Carfax for where that car has been.

More recently, in my search for my teen's car, I found a deal that was too good to be true. A great price on the model and color we wanted, and it was even a recent year edition. When I went to inspect the vehicle, it looked great on the outside. But I'd learned since my experience with the Altima about what to look for. So I took my flashlight with me and started inspecting the wheel wells and underneath the car. I saw signs of overspray and also missing rear bumper cover rivets. It was clear to me the rear bumper cover had been replaced, and you would only do that if you'd been in a crash.

Now, you can buy a car that's been in an accident and you might be fine. On the car I looked at, it really was clean in all other respects, so if the accident was cosmetic only, then it would have been a chance to get a good deal. But again, you never really know, so in the case of my teen, I didn't want to risk that the safety rating was in any way compromised by this unknown.

The Hard Part: Value Assessment

Once you have found a vehicle you like, it's time to tackle the hard part. You'll have to see the vehicle and then start negotiations with the seller. I'm assuming in this article that the seller is an dealer or institutional seller and not an individual.

Mental Preparations

Before going to see the vehicle, I'd suggest some mental preparations. Steel yourself against your own human nature and harness your discipline and patience. You may think you are about to deal with jackals that have every advantage against you, but that's only partially true. If you can affirm these positions, you will have some advantages of your own:
  • I'm not in a rush to buy a car.
  • I'm not going to buy the same day I check the car out
  • I'm not going to show any emotion, whether I like the car or not
  • There are a million used cars out there, I am not desperate for this one
  • I don't care what the dealer paid for this car, I only care about what I am going to pay
  • That car isn't getting any younger, the dealer needs to move it more than I need to buy it
Of course, to be able to think the above thoughts, you have to be in a position where you really don't need the car. The old adage about job hunting says that it's easier to get a job when you already have a job. The same is true for cars. It'll be a lot easier to get a deal if you don't need the car.

Is there risk that a nice car will get sold before you decide to buy? Yes, especially if the car is a popular one. But if a fair deal is more important to you than a specific model or color, then that's the risk you take. Even then, the point about there being other used cars out there is true...and there's always going to be more inventory tomorrow. 

The Inspection

Contact the dealer and arrange for a visit. TIP: try to go see the car near closing time for the dealership, like maybe an hour before. I'll explain why later.

You'll want to analyze the vehicle carefully. If you have a friend that's a mechanic, bring him or her along. One of the other sites I found had some interesting advice, and said that when you do your review of the vehicle, show no emotion and say nothing, just be very businesslike and focus on the inspection. The only emotion you should show is some irritation if you see blemishes. Bring a flashlight to help with inspecting dark areas. Here are some of the things I've learned to look for:
  • Overspray: this is excess paint that appears in places it's not supposed to be. It's a sign the car has seen damage and needed cosmetic repairs at a minimum, and possibly more. If the Carfax was clean on this vehicle, maybe it was just a minor scrape, but it's a big unknown either way. Look for overspray in wheel wells, the underbody, any of the body panels, and sometimes even a bit on tires; anywhere that the painter may have done a poor job of masking.
  • Missing screws, rivets, or clips: As in my story above, retaining parts that are missing are a sign someone had to remove a part. It is possible that a piece just fell off, but pretty unlikely that multiple parts would, and in my story, the rear bumper cover plastic rivets were missing from all four holes where they should have been. That's no coincidence. Sometimes people in the car industry are sloppy and these are the signs they would leave after having to do body work. 
  • Asymmetrical features: parts on the left and right sides of the vehicle should match. A cover on a taillight should be the same color and material on both sides. Also check things like mud flaps and wheel bolts. If anything's missing it's something to note when you decide if the price is fair or not.
  • Tire tread: a used car probably won't have new tires, but you don't want bad ones that are unsafe or that'll have to be replaced shortly after buying.
  • The engine compartment should be fairly clean and everything should be intact unless it's more than five years old and then there might be residual chemical deposits or dirt. In most cases, models from institutional sellers are going to be more recent (within five years) unless it happened to be in really great shape.
  • Look for signs of a dirt line on the exterior in wheel wells or inside the bumper cover (modern cars mostly have a plastic bumper cover on front and back) or discoloration near the floor of the interior that might indicate flooding.
  • Scan the exterior panels carefully and note any scuffs or scratches. Again, you should not expect a pristine body from a car several years old, but you shouldn't pay full value for one that has significant and ugly scars either. And showing the salesperson that you even notice them will let him or her know you're not just some idiot fooling around.
  • You should be able to see the brake rotors. Verify they are smooth and don't have deep grooves in them, which might indicate the previous owner didn't take care of them.
  • If the car is supposed to have a spare tire, verify it is present, along with any tools that should be there.
  • Obviously test drive the vehicle and make sure it feels comfortable to you and has sufficient performance for your needs.
  • Verify all the electrical features are operational. Electronics can get pretty complicated on cars today.
  • Test the HVAC unit. Especially in Houston, please test the air conditioning! Also, the air coming from the HVAC unit should be odorless and not like mildew. 
Look, I could go on and on here. Cars are some of the most expensive and complicated devices you'll ever buy. But generally you just don't want to get screwed with a bad vehicle. The good news is that for the last several years, the auto makers have achieved a consistent level of quality. Of course there are exceptions, but most cars now are such that if you take care of them they'll last a while.

If you find a couple cars you like, deciding between them can be tough. The relationship between the mileage and age is interesting. You'll find a car that only a year old that has 40K miles on it, and another of the same model that's three years old but has 32K miles on it. The newer one is likely to have updated features but boy someone really drove it hard for that one year. The older one, if it was well maintained by a good owner, might be just as nice a vehicle.

The Negotiation Part 1 - Sales

Get something else into your mind before we continue: you don't know what the dealer paid for that car and the salesperson will never tell you. If he does, you must assume it is a lie. And it's ok, because it doesn't matter. As long as you don't feel ripped off when you buy, you'll survive.

Ok, so you finished your test drive and inspection. Is the dealer's price fair based on your research? If you're convinced there's nothing wrong with the car you can buy it. Yes, just buy it. It's ok, if you feel your research was good and the dealership priced it fairly, then go ahead if you need it and be happy and think nothing more of it. Enjoy the car.

But if you're not in a rush, consider saying, "I'll think about it." Car sales guys hate that noncommittal phrase. If you want to see a funny use of that line, go watch the movie True Lies. They hate the phrase because you're wasting their time. They're used to seeing you excited for a vehicle. They're used being able to sit you in their office and then spending the next four hours beating the crap out of you for the fun of it. One of my friends in the industry says that's an old tactic called "wearing you out" and boy does it wear you out. When you say "not interested" now you're the one wearing them out. So walk away.

At this point there area a couple of possibilities.
  • First, the salesperson might just let you go. Maybe that car is a popular model and there will be another buyer the next day. In the case of the car I mention above where I thought it had been in a fender bender, someone else indeed bought the car within the next week. Good luck to that buyer, I say, but I had good reasons not to buy.
  • In the case of a no-haggle dealer, they'll definitely let you walk. Texas Direct Auto and CarMax are like this. You either like the car and buy at their price, or you don't. End of story. The only way to get a deal here is to wait for a price drop if the car doesn't move.
  • Another possibility is that the salesperson will ask you to make an offer, saying something like, "What would it take for you to buy it?" or "I really want to move it, make me an offer." This means that the dealership is open to negotiation. Using your research, offer something a little below what you wanted to pay. Maybe shoot for something on the low end of the KBB range. Don't worry about being a jerk and ripping off the dealer, that won't happen. The dealers are like the house in Vegas, they always win. If they weren't winning they wouldn't be in business. I think it would have to be a rare circumstance of perfect timing and a perfect situation where the dealers would let you rip them off. It's common knowledge that the used car sales are more lucrative for dealers than new cars.
Here's probably what's going to happen once you give your lowball offer. The salesperson will talk to the sales manager about it. Then the sales manager will counteroffer or just say no. If the dealership starts going back and forth with you, just leave. They are starting the "wear you out" process. Get out of there NOW. Lie, say you have a kid at home, or say that you have explosive diarrhea and have to go. But get the fuck out of there. Don't let them waste the next three hours of your life. Remember when I suggested going to the test drive near closing time? This is why. At that time of day you have a natural reason to bail and the dealer has a natural reason not to beat you up: he doesn't like hanging around past closing any more than the next guy.

If they flat said no to your offer that's ok too. Don't act disappointed. Just say, "Thanks for your time," and leave. Go get a beer or do something relaxing, and assume that car was not meant for you. You may not know it yet but the story may not be over. You've just left the ball in their court. They know you have an interest but aren't going to pay their Internet price. But the key here is that they know you have an interest.

Now you wait. If that car wasn't meant for you move down your list to the next one. If you really liked that one that got away, relax. There will be more new ones every day. Every day. Keep searching.

If the one you liked didn't sell in the next few days, you may get a call from the dealership. "Hey, if I could get that price to $X, would you buy?" Now it's up to you to decide if his lower price is acceptable. If not, then maybe come up to a price you really wanted to pay. If it is, you can say yes. Either way he'll go talk to the sales manager again, but this time you're not there so your time isn't being wasted. When you get a call back, it won't be for the price $X he asked you about earlier. It'll be for something between $X and the original Internet price. Say no. Just say, "No thanks, but I'll let you know if anything changes," and get off the phone. You'll get another call back and they'll agree to the $X price. At $X, you're probably at least several hundred dollars cheaper than the Internet price but maybe not where you wanted. If $X seems fair to me I'll go ahead and accept, but if you're gutsy you can try to hold out for more. The fact that they called you back means they do indeed want to move that car, and moving it now for the lower price is more enticing to them than waiting for the next guy. And there will be a next guy, just like for you there will be another car if this one doesn't work out.

Woo hoo!

You win, dealership loses, right? Wrong. Remember, the dealership is the Vegas house. Those guys always win. This is about you just getting a fairer deal. That's all you can hope for. You will never get a deal where they got it on trade-in at $14K and you clever guy you, you stole it for $13K. That's not going to happen except in rare cases or if you're the world's best negotiator and poker player and you happen to hit them up on a day where the car you want has been sitting on the lot for eight months and the sales guy is going to get fired if he doesn't move it in the next day. Like I said, you'll never really know what they paid for the used car and they'll never tell you. So you can't really accurately assess how well you did with regard to price vs cost. All you have is the market value of the car as declared by NADA, KBB, and all the other sites.

Generally speaking, I believe the Internet prices are typically lower than sticker prices and shouldn't be too far off from the suggested values from those sites in your research. So if your work results in you getting a break on the Internet price, well, you've probably done OK.

It's pretty tough to know exactly how good a deal you got. There are too many unknowns with a used car. Who had it before you? What happened to the car before you got it? Is that car better or worse than the other one you saw that had five thousand more miles on it? In my case, NADA said the car I ultimately paid $11K for was worth $11.7K. Sounds great! But then KBB says the car is worth $10K. Oops. So like I said, don't beat yourself up too much over it. I know that the previous month when the dealership first listed my car, it was about $12,100. So I assume I did OK, because had I offered to buy when it was first listed and at the listed price, they'd have gladly sold it for the $12.1K...the exact same car I got a month later for $11K.

Food for thought though: how do you know some of those valuation sites aren't already in bed with the dealerships?

Another thing: you're not done yet. There's another nasty battle ahead.

Negotiation Part 2 - Finance

OK. You got your car and a break on the price. Depending on how long the sales negotiation took, you might be relieved it's over. But especially with used cars, there's some things you have to be careful of when going to finance, if you finance with the dealer.

You can of course avoid this second battle by bringing your own financing. There's a reason the dealerships get all happy when you say you'll finance with them. They're going to make a commission on the financing. They don't even keep the loan...by the time you walk out of the building at the end of it all, they've already either sold the loan to another bank (if the dealer has his own financing department through which he starts the loan before selling it) or the financing they give you is already from a third party bank.

So if you know the walk-out price for the vehicle plus tax, title, and license, get the loan on your own. You can go to most bank websites and it'll take 60 seconds to get a preliminary estimate of the rate and terms. You can then hand the dealer the payment and not worry much about the financing department. In fact, even if you plan to let the dealer finance you, still check out some loan estimates on the Internet so you can know the neighborhood your monthly payment should be in based on your expected price. This may help you later.

If you do decide to let the dealer help you with financing, you need to be ready because they might try some really dirty tricks on you.

Sleight of Hand

They'll first try to get you to think about the monthly payment, and not about the interest rate. This isn't illegal, but it is deceptive. So in an age where car rates are usually between 3 and 4 percent, you'll get stuck with 5 percent or worse. Same as with the salesperson, you keep saying no and make them work for it.

Another really sleazy trick that's not illegal but is again deceptive is that they'll not mention that the total pricing projection happens to include several extra fees that were conveniently not disclosed to you. The most common fee and the biggest one is the extended warranty. More on that later. But they'll also try to roll in gap insurance (which helps pay the loan if you can't for some special circumstances). In my latest purchase, the bastards had a new one for me. It was some total bullshit thing called "key insurance". What the hell is key insurance? I didn't know either. They said it was a few extra dollars a month to provide batteries if the wireless dongle battery wore out, or a replacement key once every five years if something happened to the key. Holy shit. That is some bullshit! If I hadn't kept balking at the financing terms he might have not ever have taken it off. This crap was going to cost me "a few extra dollars a month" for a 60 month term? No thanks.

The way you combat this is to ask the finance agent to show you exactly what is on the contract. There should be only a few required items: the price of the car, state-mandated title and license fees, and taxes. That's it. Anything else is optional and should not be on there without your consent. What they want to do is hope you have focused on just the monthly payment. Then when you agree to it they sign papers and you're locked in, having consented to it all even though they didn't specifically mention that the extra warranties are in there.

Previewing what your monthly payment should have been, is a good way to help you tell if you're being fleeced. In my case I had previewed a 60 month loan and the bank had projected a $160/month price, which I thought was probably ok and something my teen could handle. When I got to the finance guy, he offered that the price would be $225/month and I was like, WHAT THE HELL? So taking the 60 seconds to get that preliminary estimate was real useful in educating me about proper expectations. I had the finance guy tell me exactly what was on there and had him take off the extended warranty and the gap insurance but the price was still $170/month so there was something still fishy going on. I remained irritated and continued to say no to the warranties. He finally tried something else, got to a more reasonable $203/month that included the warranty. I asked him what he did and he admitted he took off the key insurance. Fuck me what a bunch of assholes these car people are. That key insurance is $300 of worthless. I can buy remote batteries at Walmart for $6 and you fucking get two of them, and in all my life I've never had to replace a car key. Fucking assholes!

You Bought an Extended Warranty? Dude, You're the Asshole!

All right. So I went with the $203/month. Wait, what? Did I succumb to the scam of an extended warranty? Yes, and extended warranties are controversial so I can understand the reaction. I have purchased extended warranties in the past and they have proven worthless, not because they wouldn't work, but because cars are well built enough now that I never had major mechanical issues, even with that Altima that had been in an accident. But generally, if I get a used car, I'm more open to them because of all the unknowns. They're typically not worth it because you have to have some major stuff go wrong for the coverage after deductible to break even. But it does provide a little peace of mind. And actually I lied. There was one time I did capitalize on a warranty when I bought a digital camera at Best Buy. Something happened and the camera became inoperable a couple months after I bought it, but since I had the upgraded warranty, Best Buy immediately replaced the camera with a new one, no questions asked. Extended warranties are not an efficient way to spend money, I get it, but when you have a lot of unknowns it's not the worst thing you can buy.

Good Luck

Monday, May 29, 2017

Memorial Day 2017: Sully

One of my previous Memorial Day posts looked at a Clint Eastwood film about a veteran. Another Eastwood film conveniently found its way to my cable channel a few weeks go, also about a veteran: Sully. This film looks primarily at Chesley Sullenberger's time as a commercial airline pilot, centering on the mishap investigation around Sullenberger's decision to crash land his airliner in New York's Hudson river after a malfunction on January 15, 2009.

And once again, there are some parallels between this movie and the broken IT industry. The aviation world, once and arguably still one of the leaders in epitomizing the best of human endeavor, has grown large and complex, characterized by many silos of power. There are of course the regulatory commissions like the Federal Aviation Administration (FAA) and National Transportation Safety Board (NTSB), pilots' unions, air traffic controller unions, additional administrations at the state and municipal level, and because there wasn't enough fun coordinating all of that (and the multitude that I don't know about and didn't mention) organizations like the Transportation Security Administration (TSA) were added.

One of the key problems in IT departments now is that the fragmentation of duties creates complication and reduces efficiency. This isn't a revelation, it's only the same thing that has driven comedic material about organizations (especially governments) for decades, perhaps centuries. This derision is well earned, with governments often characterized as a circus, a whimsical collection of nincompoops that take up too much time and too much money. In an arena with many warriors it's easy for each member of a team to forget the overriding goal to serve a customer.

Each part of the equation is fighting to be a bigger part, to preserve itself, and to survive and thrive. This is natural behavior, but can lead groups in an organization astray. The focus becomes "me" instead of "we" and each faction serves itself and the spirit of the greater organization is lost.

In Sully, the NTSB is largely painted as an organization looking to place blame on the pilot for the mishap. Although it's not directly stated in the film, it's hard not to imagine external pressures on the NTSB to find fault with the pilot. We've seen this sort of thing before, when insurance companies and their lawyers would choose obfuscation over clarity, choose a pyrrhic victory at the expense of justice. The greater good it seems, isn't served by the initiative and actions of a pilot with some autonomy to save lives, but instead by entities uninvolved in the actual incident who would save corporate dollars.

Of course this is largely just me being cynical. I have to give Eastwood credit for not openly using such obvious gimmicks to win the audience to Sullenberger's case. No, there would instead be other gimmicks used to create drama where there might not have been any. I don't know exactly how the hearings went, but in Sully, there were a couple things I didn't like in this otherwise interesting and well made film. It was hard to believe that the late breaking evidence appearing in pivotal moments of the film mirrored the timing of those events of real life. I also found it a bit too easy for the NTSB to shift so quickly from villain to adoring admirer; in real life I'd hope the NTSB isn't there to create drama but simply to approach an investigation with no expectation that the cause is pilot error or technical malfunction and to be interested in only one thing: the truth.

Ideally, each organization involved would be singularly focused on that truth, the biggest one being that not a single life was lost in the incident. Isn't this the reason for all these entities, to push for safety and standards to limit this risk to human life? Without passengers, there's not much need for a smart, experienced pilot and co-pilot, safety boards, investigators, flight crew and their training, and air traffic controllers. Every single one of the people involved in those groups should take pride in ultimately serving the same customer: the passenger.

I wish it was that way in IT too. Whether it is a new software project to create a system that improves communication or efficiency, or a hardware team that is responsible for getting the appropriate technology into the hands of the right user, or a software support team, or a business analyst, they should all understand they have the same goal. They should not be looking to put down one team at the gain of their own, or to value performance metrics relevant only to their particular space, but to understand that the production experience with technology is everyone's responsibility.

That would be a great IT organization indeed. In that world, everyone works together to provide for the one true boss: the customer. In that world, everyone is doing their part to keep the customer happy even though each may have a different role. A circus, but the good kind.

Monday, May 30, 2016

Memorial Day 2016: A war story that predicted the future of IT

I like traditional paper books and resisted going to ebooks. I like to think it wasn't because I was stupid, I certainly understood the value of their portability but I thought ebooks were soulless. I grew up in a time where a quality bound book was a piece of education and also a work of art. It was collectible and you could get an author to sign it. It had value. But eventually I succumbed to the lure of being able to have thousands of books on my phone.

I was slow to adopt audio books too, but recently started up after flooding in Houston caused my commute to become longer. And I've wiped out three books in six weeks or so, improving my rate of attacking my backlog.

One of those books was Day of Infamy, by Walter Lord. It was written a long time ago but is a detailed overview of the Dec 7, 1941 Japanese attack on Pearl Harbor. It's a great book. The Pearl Harbor attack was a surprise and a shock to many on the islands. Some people even mistook the inbound Japanese airplanes for Americans on training maneuvers.

When the servicemen understood what was going on, many moved to respond, trying to help others or pass the word. Many others tried to put up whatever resistance they could, even firing at passing planes with small arms. Getting those small arms was sometimes difficult. Lord's book mentions a couple situations relevant to this blog.

There were some instances where even amid the attack the supply clerks refused to issue arms or ammunition to the men without the proper authorization. This represents several of the themes I've noticed in our IT industry.

  • Revering Process at the cost of Effectiveness. The intransigent supply clerks had a narrow focus on their role, which was essentially to control inventory. They lacked an appreciation for the larger picture and worshiped the traditions of their gods rather than the reality surrounding them. The impetus for this thinking may have been grounded in reasonable motives (cost control, safety, accountability) but such adherence to dogma rather than the primary goal of the institution (in the case of the US military, to defend the nation) may have cost lives. This anti-pattern is often the fallout of large organizations that have naturally segregated duties for specialization and formed silos of knowledge. These silos have their own management chains and can take a counterproductive focus as they work to justify their existence. This behavior doesn't require a large organization though; there are plenty of inexperienced managers out there that can do the same thing even in small companies.
  • The Road to Hell is Paved with Good Intentions. I probably don't need to explain this one. In a world increasingly driven by people and groups that have mastered the ability to push personal interests over what's really important, you can find plenty of real world examples.
  • The Tactical Reality will Override the Theoretical Ideal. Lord mentions that in some desperate cases, servicemen took axes to the locks on ammunition cages and did what they had to do. Yes, an accountant somewhere will be very hurt by the loss of the lock, but although Harlan Ellison astutely noted that "the world is becoming a cesspool of imbeciles," people are not completely stupid and sometimes humans can be surprisingly functional. Even though I can't stand the ludicrous edicts of the Sarbanes Oxley act, SOX procedures do allow for the people who can get the job done to have provisional authority in emergencies. But I'll bet money it wasn't Sarbanes or Oxley that allowed for that, but the grunts in the trenches that fought back against the original rules.
In honor of the servicemen that lost lives at Pearl Harbor that fateful day, I wish you a happy Memorial Day. 

Sunday, April 24, 2016

Software Lessons: The Customer is always right, and sometimes so are You

Here's one of my favorite memories from a project I worked on.

Our company had acquired another company and my team had the responsibility to integrate the new company's business into our system. There were some similarities between our existing business and the acquisition's business, but as there always are when companies merge, there were some critical differences too.

We worked really hard and managed to modify our system in ways that made the new company's work possible to do in our system. One minor feature they didn't ask for stands out in my memory. I added some auditing data to one of the screens we created for them. It would track simple things like when data was added or modified. I'd always found such things handy in support, so I did what I thought was the right thing and integrated it into the system in version 1, not as an afterthought hacked in two years later.

The analyst representing the acquisition noted in one of our design meetings that this was not a feature they needed or had asked for and why were we putting it in? I explained that it would be useful for ongoing support and that it would be done in a way that would not interfere with their work and that it would not negatively affect performance. The analyst still seemed a little bothered by this unasked for feature, but accepted it with my assurances. Note that by both some project management standards and agile development purists, this would have been considered a violation of the rules. In those worlds, I would have had to remove the feature (although it could arguably be a fair inclusion under what business analysts call a non-functional requirement).

Fast forward to about six weeks after the system went live. Things are going relatively well. The hard work in up-front design has mostly paid off and we had only couple notable bugs reported. Most either had workarounds or were fixed within the first post-implementation release. But then we get an interesting report from the same analyst that earlier had said there was no interest in the audit feature.

"We notice that the audit data doesn't seem to be working right. We expect the detail level items to be time stamped with the date of creation and they appear to be picking up the date of creation from the header level data instead."

I wanted to say, "Wait. Wow. Really? You are reporting a defect on a feature you told us you didn't want and wouldn't use?" But of course I said instead, "We can change that. We'll set up a change request and plan this for a future release."

Clearly, they'd been using the feature to see when data was created and/or modified and by whom. This is actually very useful reference information for both the support team and for the users. There are any number of situations where it's handy to know when and how something was changed, especially if that change is to foundational data that affects downstream processing. Unfortunately, not all systems track that information and many that do fail to make that information accessible.

The point here isn't to gloat that I was right. The point is that experience with support and systems can yield valid input into shaping subsequent systems. During design, there are many people of different experience levels and different personalities and perspectives trying to influence the product direction. Sometimes you, as the person that might live with the system after go-live, must find ways to get your ideas in place, even when traditional roles of expertise don't agree.

In theory, the International Institute of Business Analysis (IIBA) pays respects to the non-functional requirements, that is, requirements defined by the engineers. In reality, project managers and less worldly business analysts will overlook them in the interests of increasing project velocity. They don't have to support the beast for the next ten years, you do. So stand up for things that are reasonable and valuable for your team.

The customer is always right, and sometimes so are you.

Saturday, April 23, 2016

Software Lessons: The Long Tail

One of the top misconceptions I've witnessed in IT work is that many view a software project as a discrete component. They assume that the costs are static and once a piece of software is in place, the costs end or are minimal. They will look at the cost to build and implement on Day 1 as the only cost.

This is so wrong that it befuddles me how often this mistake is made. The "long tail" of a system's life, comprising the care and maintenance of the application, is likely to be far more expensive than the cost of the initial implementation (assuming the software is useful enough to live for more than a few years).

This is especially true if the quality of the application is poor due to corner-cutting in the creation process (and given the common emphasis of deadline over quality, it happens a lot). This is why developers and project managers are often at odds. The project manager is working to a milestone and then moving to the next project, but the developer likely has to live with the completed product for a longer duration, suffering with that application's deficiencies. I've discussed this before, where such scenarios create soul-sucking and inspiration-sapping work that is menial and tactical in nature rather than strategic. Many of the people at the high end of these mistakes are not held accountable for these problems, due to the way the typical reward system's metrics work.

Change will require a major shift in metrics, and it must come from the top.

Thursday, March 10, 2016

Shading rows for readability in a PowerBuilder datawindow

Just a little almost-useless tech note for myself.

I really like when developers do little things that make my life easier. Sometimes you'll see reports with a lot of data get shading to help your eye track the row you're reading.

I figured out a way to do this when I was doing a lot of PowerBuilder work. Apply an expression to the color property of the detail band of the datawindow. It's fairly easy to do if you just want to alternate every other row between two colors, but it gets trickier if you want to do it with more than odd or even rows.

Here's the code to shade every alternating set of three rows between light grey and white:
if ( mod(
          if( mod( getRow(), 3) <> 0
            , int(getRow() / 3)
            , int(getRow() / 3) - 1
         , 2
         ) = 0
   , rgb(255, 255, 255)   //white
   , rgb(233, 233, 233)   //grey

What this code is doing is using the modulus function to analyze the remainders of the row number divided by 3 to reduce it into one of two categories, grey or white.  In a division by three, the remainder would be 0, 1, or 2. If it's 1 or 2, we know it's the first or second row in a block of three rows. If it's a zero, we know it's a last row in a block of three rows.

But we want that third row to be included in the batch of three rows of a single color, and to alternate with every other set. So here's where the PowerBuilder INT function helps. We divide rows 1 and 2 by 3, and get a fraction, but the INT function simplifies them to the largest integer below the given value, so for rows 1 and 2, they INT function returns 0. But for row three we don't want it to fall into the next grouping so we subtract 1. Then we MOD these results by 2 to get the binary result of either a 0 or 1.

This makes our formula generate these values for the first several rows:

Row  Nested_IF  Outer_MOD
 1         0       0
 2         0       0
 3         0       0
 4         1       1
 5         1       1
 6         1       1
 7         2       0
 8         2       0

See how that works? It's fairly simple for the computer to process, but easily breaks each row down into either a 1 or a 0, and at that point it's pretty easy to wrap the whole thing in an IF statement and just assign the color.

I really only wrote this so I wouldn't forget it if I needed it again, but you're welcome to use it. If you're feeling "old school" you can use a light green instead of grey for the colored rows. And if you're a brainiac that figured out a way to do this with even less code, please post in the comments, I'd be happy to find an easier way.

Saturday, July 04, 2015

Book Review: A Technologist's Guide to Career Advancement by John Schneider

I saw someone mention this book in an article comment and bought it as it looked interesting. I also posted a review at Amazon but wanted to write a more detailed review here.

Advice from an IT Success

John Schneider has had an eventful and successful career in technology, working his way up to executive levels. This book couches itself as a career guide specifically for technology workers. I didn't find a whole lot that was specific to technology; there is a lot of great advice in this book for pretty much anyone. Schneider writes with an easy style and uses a lot of humor and the book is a fairly quick read. I think it's probably a good book for most people who want to know how to stand out in any industry, although it's likely a bit better value for younger people who have time to put into play the things Schneider recommends.

IT People Have an Edge?

The one conceit Schneider carries that's specific to tech workers is that you can do most of the other jobs in the company but the other people couldn't do yours.

This is often true but I would prefer that it not be presented as an absolute, because I've met my share of IT people that probably couldn't do other people's jobs, much less their own. And I've met smart folks outside IT that could be great in it.

But I do like the gist of what he's getting at because it matches an observation I've made about experienced IT teams: they represent an interesting foci of business and data. That is, they are people that understand the technology, but have also probably picked up knowledge of the company's business and its clients. They also are positioned to recognize the gaps between departments that need to use the same data. That puts them in a unique position to solve problems and improve the company.

As companies grow, they naturally tend to fragment and become a collection of silos, each narrowly focused on its specific function. This is a dangerous structure that compromises efficiency, ironically the very thing that specialization is supposed to provide. It becomes inefficient because people begin working in a vacuum and lose insight into why a task is done and how it affects others downstream in the process ecosystem. Communication tends to suffer too as people get busy doing their own things and this gradually reinforces both the distance between groups and the calcification of process quirks that might have been workarounds for something that was a problem once but might have since changed. And without an oversight to recognize this condition and an agent to promote improvement, companies (even if still solvent) eventually suffer atrophy. Teams become political and defensive, trying to justify their existence and role, even if they'd be better somewhere else.

The agents of change would ideally be managers and business analysts. But we know this isn't the case; higher management typically does not listen to the things their subordinates are telling them. They've evolved into selfish entities that cater to self preservation and shield themselves behind barriers of elitist cliques and faulty assumptions that they, and not the customer, are the profit center.

Killing Sacred Cows

Schneider isn't shy about challenging conventional career wisdom.

He disagrees with some things that are considered industry best practices, notably the advice on accepting a counter offer when resigning. The general rule is that you don't accept them. If there were fundamental problems at the job that caused you to want to resign, are they really going to change if you accept the counter offer? Schneider does acknowledge that if a workplace is really dreadful you should just leave, but then goes on to say the things you've heard about taking a counter offer are "BS" and taking one is just fine. I too will concede that if the parties involved are mature consenting adults and not children that can hold grudges, it might be ok.

But people are human beings and both the company, the managers, and your peers will remember what's transpired (you might try to keep it a secret, but things have a way of getting out). Ultimately, if you had to threaten to leave in order to get what you want, is that really the kind of place you want to stay? These are legitimate caveats to accepting a counter offer and Schneider is perhaps a bit to flippant about them; in addition when he calls them "BS" he doesn't really provide an argument about the specifics of why they are.

He also seems to value the effort one could put into acquiring certifications. I think a lot of experienced IT staffers will tell you different things here. In my career I've managed to do well without certifications. Schneider feels they will be useful in helping you advance in the organization and discounts the value of seniority; that may be true in some companies. But my personal experience is that certs are best as differentiators in getting hired, not promoted. Once you're in an organization, promotions are likely to be based on a combination of things such as performance, politics, and yes, seniority. Often, very much about politics and seniority.

Higher Education

He recommends getting an MBA; not bad advice, but there's more to this than meets the eye. He casually shoots down several excuses people might use to not get one when some of them are actually really good reasons. Cost, for example. If you want to go to a prestigious program, it might cost you the amount of a nice house; I would not so flippantly disregard this barrier. And, in my experience, the value of an MBA is like that of a cert. It's probably a great way to get your foot in the door, but advancing beyond that point will depend on performance, politics, and well, seniority, though I do see a distressing trend today to put young and inexperienced people into positions of power where they can destroy companies largely because they have an MBA and are experts at cost cutting, Mark Hurd-style. So maybe Schneider is right about that after all.

Are Things Different Now?

I don't doubt that Schneider is a successful and brilliant guy and probably a good boss too, if he practices what he preaches. However, I couldn't shake the feeling at times that he's led a bit of a charmed life. His thoughts about certs, seniority, and MBAs make me feel that he's been fortunate to traverse most of his career through meritocracies. But I'm certain I'm not alone as an IT staffer that's recognized technology workers have become the contemporary blue collar workers of the world. IT shops are seen as costs, not strategic components, by most companies. As a result IT people are constrained by a very thick ceiling barring them from the highest leadership positions (roles open to operations, sales, engineering, marketing, and even accounting) where they could bring their blend and breadth of business and systems knowledge together to truly help a company forge strategic initiatives in intelligent cost cutting rather than mere layoffs and the practice of being cheap at the expense of efficiency.

All this to say that while Schneider's advice is still overall very good, it may have had more effectiveness before IT departments evolved into the bastard stepchildren of a companies today, a time before PMP's started telling us to forego innovation for smaller and more easily measured changes. A time when workers were allowed to think.