The Linux Adventure Part 5: Everything is Broken

Ye gods.

You know all the stuff I wrote in the last five blog posts or so about the NAS unit? Well, it turns out that while it worked, side effects broke some the unit's functionality. First, the unit's network light would flicker and then it would cease to go to sleep, a critical function for energy savings on a 24/7 appliance. Then I also noticed the Diskstation would not shut down or restart when given the manual command to do so in the software.

After much gnashing of teeth and many face palms, I learned that bootstrapping the unit caused the issues. Synology generally is quite liberal about bootstrapping, even including information on how to do it in its official support resources. However, the operating system software, Disk Station Manager (DSM), gets updated regularly. Usually, this is a good sign that shows a company hasn't abandoned its product and is actively supporting and improving it. However, it also means that the environment running the station is a moving target. And by also allowing bootstrapping, Synology has led me into the very bear traps I see corporate IT shops sticking their balls into every day.

Young Profession, Old Debate

In the corporate IT world, there's an established debate about "build vs buy". Do you build the software you need from scratch or do you buy an off-the-shelf package? The debate usually has these points:

Build

• PRO: Software is customized to your needs and way of doing business
• PRO: You have complete control of the code
• PRO and CON: You have complete responsibility for the code
• PRO: Proprietary business knowledge is institutionalized in the code
• PRO: Enterprise processes are enforced by the software globally (barring proper implementation)
• CON: In-house development is expensive
• CON: In-house development often requires non-software shops to have proficiency in software development (bigger con: in reality most IT shops have at best mediocre competency).
• PRO: Properly implemented, a custom software team can more rapidly change the software than a major vendor can or will

Buy

• PRO: Buying someone else's software is less expensive because you don't have to have in-house development resources and licensing is cheaper than development. That's the theory anyway...in reality I'm not convinced it's less expensive, but I agree that writing a check monthly is probably less work than having to account for a staff of developers and all the extra HR overhead.
• PRO: Buying means you just use the software and the vendor handles development and support. Those of you that know better can laugh now.
• CON: Your business now does business the way the vendor's software wants you to, not necessarily the way your business people want to.
• CON: There may be limited capacity for customization of the vendor software to implement proprietary strategic processes
• CON: Reality shows that for core business (not commodity items like word processing or spreadsheets) buying is still expensive and implementation missteps often negate any cost advantage
• PRO: Certain industry standard processes, if implemented well, can be a part of the vendor software
• CON: It is rare that all companies do business exactly the same way
• PRO: Vendors may be able to capitalize on integration with common third party packages for things like accounting
• CON: You may think that a vendor is more dedicated to the principles of good software design and best practices and therefore will deliver stable, efficient and intuitive high-quality software just like the kind you find on Apple computers. The truth is that some shops may be very good, but most are made up the same knuckleheads that soured you on the "build" approach.

We Want to Have our Cake and Eat it Too

In the 80's and 90's it was common for companies to take a "build" approach. Software was new and exciting and there weren't many vendor packages available, so the typical IT guy said, "Oh, it's easy, all we have to do is..." and much spaghetti code was born.

Later companies started realizing what a mess it was to maintain crappy systems. Prodded along by drinks on the golf course and rides on corporate yachts, they decided to buy instead. But they made a critical mistake. They still wanted to do business "their" way and that required changing the software they bought. They wanted the best of both worlds but ended up with the worst of both worlds by buying and then heavily customizing.  Oops.

Now they had vendors that wouldn't or couldn't respond quickly to changing the software to fix or add desired functionality. Their in-house customizations did address some of the core software's gaps but built by inexperienced developers, proved cantankerous and bug-ridden and the users hated using them. And there was another issue: when the vendor had a new version, upgrades were that much harder because they could break the customizations. Many would suffer nervous breakdowns during this time, but consulting companies would happily offer help in exchange for a chunk of the company's life savings.

Holy Crap I did it Too!

That brings me back to the NAS unit. I got it initially so I could store my photos, documents and music in one location instead of having them dispersed on four different PCs and dozens of other flash cards and portable drives and memory sticks. I could have rolled my own by putting together a simple Linux server in a low profile case, but I wanted the off-the-shelf solution that would let me plug-and-play. I wanted the benefits of commodity.

But like the corporate idiots before me, I got greedy and wanted this wonderful Linux server to do more. So I bootstrapped. I customized. And I encountered an incompatibility with the latest version of the NAS software that doesn't work when you also have the NAS change the default shell to bash from ash. It caused other commands in the script to fail, so several services such as the sleep function and the audio server and photo server ceased to work. The unit would not even heed the manual shutdown or restart commands. Ugh.

I suppose I'm not quite as stupid as the so called "leadership" which commits to decisions that really create hassles for thousands of people and ultimately cost billions and billions of dollars. My suffering is confined to just me; honest men wouldn't have it any other way. But I do feel some embarrassment at having made a similar mistake.

Sometimes It Takes Two to Mess Things Up

I had help though. For the first year I owned the Synology unit I was pretty happy with it. I still am, when it comes to the basic functionality of the unit. However, when I spend money on something I expect it to work, not to be brittle like the rest of the software out there. The support forums for the Diskstations are filled with people that have similar problems as mine, and some even from folks that did not bootstrap. It appears that the regular updates to the DSM software can be risky, which shouldn't surprise me, but the level of dramatic errors and functionality loss that can occur do. This is Synology's hardware, not mine, so they ought to be able to release a beta that doesn't crush functionality. This excellent thread [serverfault.com] shows though that apparently Synology's developers took several short cuts and made some sloppy moves in building their software and products.

In my case, I had to remove the lines I added to profile config to launch the bash shell and allow it to remain in the default ash shell. Now the Diskstation again responds to the manual shutdown and restart commands. After reindexing, Audiostation is back to working status. However, the sleep behavior is still broken, and now the next thing for me to try is downgrading the DSM software (currently in version 4.3 beta) back to perhaps version 4.2. The DSM front-end software warns before doing DSM updates that the DSM software cannot be rolled back, but thanks to the enterprising user community, there are ways to do it.

I'll be working on downgrading the software. I will probably lose the GUI-based task scheduler since that was part of the 4.3 beta, but I may still be able to access crontab in ash, and create the scheduled job that way. And having the Diskstation off for a bit isn't so bad; it'll be nice not to worry about those Internet pinheads that keep probing the machine. In the meantime I'll continue to use my laptop for some of my Linux needs.

The Linux Adventure Part 4: Putting the machine to work

All right. Moving the vpnc command to the main script has indeed worked, though I do need to quickly get onto obfuscating the password so it doesn't sit visible in the config file.

Once I did that, I confirmed it ran ok, connecting to the VPN, running the database copy, and then disconnecting from the VPN. All ran well.

To automate, I utilized a new feature of the Synology Diskstation's DSM software, version 4.2. There's now an integrated Task Scheduler. Interestingly, it doesn't appear to be a simple GUI interface to crontab. Instead it has some proprietary commands and probably data structures. It's also poorly documented. The help file explains what the various parts of the Task Scheduler screen are but doesn't have Synology's usually good tutorials on operating the feature. DSM 4.2 is in beta I believe, so perhaps the documentation will improve when the final product is out.

In any event, it's fairly self-explanatory to set the custom user job up. You give it a name and enter the command exactly as you would enter it at the command line interface, and then set a frequency. You can also run on demand at any time from the GUI.

I don't like that there's no record of tasks in the logs when you run them. Perhaps that will also be improved in the final release. But as it stands, when you run a task it doesn't look like anything is happening and none of the task's output is displayed anywhere.

You have to run this line from the terminal session and it will give you information on the scheduled tasks and what their last run status was:

/tmp/synoschedtask --get

Aside from the /tmp directory being a weird place to put a main feature, this line will return a list of the scheduled tasks and their configurations, along with a last run time and status.

At this point I have a boat load of error handling and feedback features to add but can now start taking advantage of automation to have this thing run automatically every day.

Synology Diskstation Security Tips

If you have a Synology Diskstation and you're only using it for local hosting of files on your local lan or wifi network, this may not be as critical.

IP Auto Block

But if you've opened up the unit to accept connectivity from the outside world via the Internet you would be well advised to regularly log in as the admin and review the system logs. I didn't notice much activity at first when I allowed remote access to the Diskstation, but recently I started seeing regular attempts to login from unknown sources. Several each day like these:

It's really unnerving to see that. I'm not naïve but my server is literally a tiny speck of nothing in the universe and has nothing of value to anyone but me. Yet the universe must be filled with pinheads who have nothing better to do than try and hack random IP addresses. Actually, it's highly likely the login attempts aren't being done by a human but by a bot that's already found its way onto other servers and is just probing.

The Diskstations have a feature called IP Auto Block. You can find it in the Control Panel of the Synology server admin tool. Turning it on will make the Diskstation automatically block any IP communication once it has witnessed five failed login attempts. I highly recommend you turn this on. Since turning it on my Diskstation has regularly been blocking several addresses each week. Ye gods, is there no honor left in this world?

Antivirus Essential

Also, Synology offers a utility called Antivirus Essential that you'll find in the Package Center under Security. It's a free anti-virus tool. So far I haven't had issues with viruses getting on the Diskstation but I'd also recommend installing this package to help ward off some potential problems.

(Update Jan 2015) Maintain System Updates

People complain all the time about Microsoft's regular patching. Unfortunately, given the propensity for people with too much time on their hands to be constantly hacking systems, I think Windows users must keep abreast of updates (even if some of the updates are of questionable quality!). The same is true for the Synology gear. I recently posted about issues [burningends.blogspot.com] I had with the Diskstation updates, and it turned out my device might have been compromised by some bitcoin hack. So, although updates have sometimes wreaked havoc with my Diskstation's sleep function, the reality is that part of my security responsibility is to keep up with updates.

The Linux Adventure Part 3: A Path to Success

Installing MySQL on the Diskstation

Continuing from the last installment: there are several items I listed on the to-do list that actually were already done. I didn't need to install MySQL on the Diskstation as it is already on there and enabled either by default or when I first configured it at setup.

Installing Bash on Ash

From the last installment, I had already installed the ipkg package management software. The command ipkg install bash will pull down the bash shell and install it on the Diskstation.

Test Manual Run of Existing Script

In preparation for the script test I first ran /opt/sbin/vpnc . This opened a connection to my client's VPN.

I then copied my existing bash script over to the Diskstation. But ran into several issues when trying to execute it. I had to relearn simple things like making sure the path was a part of calling a script unless it was already in the working directory (pwd shows current working directory).

I also had to learn new confusing things about Linux since a few confusing things weren't enough.

I made bash the initial shell by adding a few lines to the root profile to launch the bash shell. You can find these on the Synology Google hits for "Synology bash" but even after now getting the bash prompt when signing in (bash-3.2#)I still had to use special syntax to run the bash scripts through the bash shell.

In other words, I installed the bash shell, got it to be the default when logging in, but still the scripts will try to run under the Diskstation's native ash shell [Wikipedia.org]. Got that? Thank you so much Linux. So upon initial attempts to run dbbackup.sh I would get beautifully intuitive syntax error messages like this:

Line 15: syntax error: unexpected "("

Using parenthesis when defining variables indicates you are using an array to store the value. I don't know many languages that don't support parenthesis or arrays, but because ash is designed to be super lightweight it doesn't quite have all the same functionality as bash. Here is a thread [busybox.net] explaining that ash doesn't support arrays. You can take a look at the documentation for ash [in-ulm.de]. It's pretty amazing considering its tiny footprint.

But rather than look for ways to reinterpret the script in ways compatible for ash, I found some other people had similar problems. Ipkg installs bash under /opt/bin and you have to execute the bash script almost like a parameter to the bash command. Like so (where the working directory is the location of the script):

/opt/bin/bash ./dbbackup.sh

This now made the Diskstation try to execute the script, but I was not home free yet. There were some calls in the script to commands such as mysqldump and mysql, and both of these also needed to be prefaced with their directory locations. So I updated the script by adding /usr/syno/mysql/bin to the front of those command calls.

Now the script started to run, but I got one last error before it would complete the backup. The Diskstation told me it couldn't find the destination database on MySQL localhost where I would be copying the source database to. So I launched phpMyAdmin (a Diskstation natively supported package) and connected to the localhost server, then simply created an empty database for the backup.

It Works! I've Finally done Something that Works!  

    - Doc Brown (Back to the Future)

Now I ran the script and it worked just as it had on my laptop, reaching out via the VPN to the client database, grabbing the tables it needed, and copying them to the Diskstation. Awesome. Additional tips:

• The Diskstation's MySQL installation is found at /usr/syno/MySQL/bin
• The Diskstation's MySQL localhost server stores databases at /volume1/@database/mysql 

Now I have opened up some options and benefits for this task. I have:

• Multiple ways to back up my client's database
• mobile notebook
• desktop at home
• remotely connecting to the Diskstation
• Increased my knowledge of the Diskstation and my love for Linux's idiosyncrasies
• Gained a backup of the backup (the Diskstation utilizes mirroring)

There are a few things left to do before the process is fully automated. I need to add the vpnc commands to the script if possible so I don't have to do those manually. Also, add error handling to that if something goes wrong the script will correctly tidy up after itself, closing the vpnc connection and exiting the script.

The Linux Adventure Part 2: How to get VPNC working on a Synology DiskStation 212

This is more of a personal note post to help me remember how the hell I navigated all this arcane Linux shit before I forget it.

In the future, all storage will be a personal NAS device

Have you used a Network Attached Storage device yet? It's basically a personal server in the internet cloud. Except that it sits on your desk at home and you can totally control the whole thing. You can buy ones that will do disk mirroring so you always have a backup, and these days you can do all kinds of great shit from them like stream music and movies and photos. People also run security systems off them and host web sites. If you're willing to tinker a bit with whatever OS is running the thing you can do some really cool shit.

What I'm doing right now is trying to get my Synology Diskstation 212 to automate a task I've been doing by hand for a while. I have a Linux laptop that has MySQL and vpnc installed. I connect to a client's VPN and then run MySQL commands to do some database backups for them. I've got the scripts down to where they're pretty reliable and I just have to handle the connecting and the launching of the script, which then runs through a list of the tables it wants and takes care of copying the data. It only takes me a few minutes but having to boot the laptop daily gets old. You Linux guys can laugh at me, but sometimes fiddling with Linux gets old too.

You want to know why people still use Windows? This is why. All this crap below should not have taken me the time it did. The arcane error messages made me think I was still using early versions of Oracle. Then there were the labyrinthine paths to find various executables and packages and different kinds of packages and worries about whether commands and directories in my version of Linux are the same as the one in the documents I found...yes, this is why Windows with all its faults is still useful. Being an explorer is fun when I'm doing something leisurely, not when I'm just trying to get some damn work done. And yes, I do appreciate that Linux doesn't cost anything, so all the "fun" I had was for free!

Anyway, I found out that the Synology Diskstation runs a lightweight version of Linux but that it should be able to handle what I'm needing. It took me a while to finally get vpnc to work, but here's what I had to do:

1. Install a tool that lets you communicate with the Diskstation.
1. I was able to navigate the Diskstation's files with the great free tool WinSCP [winscp.net]. Its excellent GUI made moving around easy for Windows users. However, when it comes to the command line, WinSCP's built-in terminal doesn't handle interactive sessions, and is mainly just for submitting single commands.
2. To augment WinSCP you can install Putty and have WinSCP point to it. I happened to have another great free tool called Tunnelier from Bitvise [bitvise.com] installed so I used that.
2. Reference this guide [Synology] on enabling modifications and the command line interface on a Synology station.
3. In the same guide linked above you can also find information on bootstrapping the Diskstation.
1. You will have to do some legwork to figure out what processor your Diskstation is using and install some software.
2. Bootstrapping allows you to run a package management system [wikipedia] called ipkg [wikipedia].
3. Packages are software modules ipkg loads on the Diskstation. There is one for vpnc, which is what I ended up using.
4. This link [dd-wrt.com] and this one [codrspace.com] are to some quick ipkg tutorials that helped me update the ipkg list of packages and find and install vpnc.
1. Originally when I tried to install vpnc via ipkg install vpnc, I received an error "Cannot satisfy the following dependencies for vpnc: kernel-module-tun".
2. To resolve the above I used options on the ipkg command to bypass dependencies and force the install: ipkg -force-depends install vpnc
5. I confirmed vpnc installed by running ipkg list_installed
6. I tried running the command vpnc and got another error: vpnc: can't open /dev/net/tun, check that it is either device char 10 200 or (with DevFS) a symlink to ../misc/net/tun (not misc/net/tun): No such device
1. To resolve the above, I found that I had to install a kernel module called tun using this command: insmod /lib/modules/tun.ko. See also this.
2. Verify the tunneling module is installed with this command: lsmod | grep tun
3. Ok, so you might ask, why didn't I install tun first, then put vpnc on? I did try that many times on the journey but either I didn't do it right or Linux just has a sick sense of humor. I'm betting on the latter.
7. Finally, I was able to then go to /opt/etc/vpnc and add a .conf file with my vpn settings.
8. Now when I tried vpnc it found the vpn and I was able to enter credentials and connect.
9. Then I yelled "Fuck yeah Linux is great after all!" (Sorry, I have been chasing this issue for months and was happy to finally get it).

Here's what I am going to do next:

• I have to get MySQL up and running
• Get the bash shell going on the Diskstation
• Test a manual run of my existing scripts
• Further improve the scripts to automate the vpn connection
• Set up some kind of chron job to schedule the script
• See if I can get the Diskstation to email from the script
• Further enhance the script by having the daily job email a status

Corporate America You're Doing it Wrong: Part 3

Employees

Next up, let's talk a little about your employees. You know those people right? They're the ones killing you with payroll costs that you're trying to replace with robots.

If you believe Drucker's correction that there is only one profit center (the clients) then you and your employees are all about delivering your product or service to the client.

We're not Standing at an Assembly Line Anymore

A lot of managers think, rather selfishly, that all that matters is that the manager make the deadline and that the employees are automatons that simply are there for the paycheck and really don't have human aspirations to learn or advance. The manager is really only worried about one thing and that is to uphold the metrics that his bosses judge him by. This is usually, get X task done by Y date. So everything is done to make a date even things like cutting corners or knowingly producing less than a quality product.

I understand this is the way the world works. But if management and executive management is not going to care about anything else but the lowest common denominator in the world of work, then you don't get to whine when I call it out as selfish, unprofessional and inefficient. 

It's already been documented elsewhere that a happy employee is a productive one. It's also true that oppressive tactics can yield productivity gains, although I don't know many people that would want to work for Nazis cracking whips, and thus those managers will always have to deal with the overhead of turnover (as long as there is a competitive free market that employees can navigate).

Your Job is More than Looking at a Gantt Chart

But how do you fix the employees in the product-client-employee triangle? You start by looking in the mirror instead of pointing a finger. You're management. Employees will look to you for direction. Management should be about more than dates. It should also be about leadership, but the bulk of those in management roles are falling short here.

Not that I don't appreciate the challenges of management. There is a huge one as we deal with the integrating global economy. Outsourcing, cost-cutting and frantic desperation aren't totally inconceivable when faced with trying to be competitive with the Indians and the Chinese. But I can't help but feel that I'm better off now as an IT consultant rather than an IT employee. I'm doing the same work but actually treated better. That's a horrible thing. Your employees should feel proud to be a part of your company and feel that there's always something more they can do for the customer and are excited to do so. That's not the case, especially in IT, as I've discussed in posts past.

I guess there's no easy solution to the employee puzzle. But I believe that if management took a more encompassing approach to its job, it would go far beyond dates. Treat employees like, well, human beings. They will take direction and they will do your work, but if you want more than just a robot you have to develop it. Some employees are really good about this, and they will demand various conditions and sometimes when they're proven as effective workers you meet some of their demands. There's a whole contingent of them that are also not quite as vocal but still appreciate the concept of continued skills development and growth opportunities. And this is the humble group that will often go the extra mile when you need it and not complain too much, at first. Ultimately though they would like to be included in the plans, to have some sort of career path.

What Employees Want: Money is Nice, but so is a Plan 

But wait, how do we make it so that everyone can advance if there's ultimately only one CEO spot available at time? Well, I think people are smart enough to recognize everyone can't all be in the limited spots, but there's a myriad of roles people can play through different layers and columns of the organization. Be open to the idea of cross-training and gaining the benefits of creating a workforce that is less narrow; where someone can understand HR, accounting, operations, and IT. There are already proven benefits when a developer has a good grasp of the business; think of other synergies that could be gained if they also understood data pathways of HR and operations and perhaps streamline some of that data and utilize it in multiple ways. At some point you'd have a heck of an analyst/programmer/consultant that would do great things for the company, ultimately to more efficiently serve the customer. The point is that continuous intellectual stimulation is a huge part of motivating people, especially knowledge workers. Promotion is nice, when it's warranted (and I'm not sure it's always warranted when it happens) but if you can't give a promotion, at least you can fight stagnation.

Quality is the Bulwark Against Production Support

The other thing you can do is something I've touched on before. Push for quality, not just dates. In IT if you have lots of production support, you might think it's because the users are stupid or computers are unreliable. Both of those might be true, but the heavy production support is actually an indicator that you're paying for the sins of the past. All those corners you cut and sloppy crap you pushed into production, all those times you gave the business analysis phase the short shrift, all those times you cut training, or barked at developers to "do it fast rather than right" have come to haunt you now with systems that are counter-intuitive, brittle and cantankerous. The sad thing is that not only you are paying for the sins of the past, but it's likely these were someone else's sins. And we'll continue to make these same mistakes again and again in a problem endemic to the IT industry because of the decision-consequence gap. It's a gap that can only be closed by a managerial intervention that's probably never going to happen.

Stop Thinking in Binary

We also have this damaging philosophy in IT that there are two kinds of people, project workers and support workers, and that they are to be separated by a steel wall. That is a terribly narrow-minded and broken view that only leads to more production support. It contributes to the decision-consequence gap because the gap isn't just about management; developers too tend to view themselves in the projects vs support light. The "project" side tends to get more respect because those projects involve creating new code. But having been on both sides [programmers.stackexchange.com, at least until someone deletes the thread], I believe you become a better developer when you are forced to see how users are receiving your product. That means rotating among both the project and support sides. One flaw of only being on the project side is that you don't learn from your mistakes because, thanks to the decision-consequence gap, you don't have to see them. That flaw leads to another: you make the same mistakes repeatedly, and you're constantly looking to try new technologies rather than perfect every day techniques [burningends.blogspot.com]. Again, developers left to their own devices will continue to game the system and stay on the irresponsible "I can have sex and never worry about raising the child" path.

Management that is smarter than this will find ways to change this; and not necessarily with a whip. Find ways to incentivize creation of more rounded developer and a fairer distribution of work types.

You're Doing It Wrong

Of course I'm not holding my breath here waiting for a great manager to fall from the sky like Thor and fix problems with his magical PMBOK hammer. But if the industry won't change, then I'll stay a consultant instead; playing the game instead of letting it play me. I'll help cover for your past sins, but it's going to cost you. You can outsource, but that might only make it worse; you'd better be gone before your boss finds out it's not really cheaper to do it that way. I'm hope I'm not being unreasonable here and to be fair, employees have to come halfway too and accept that work is sometimes about doing stuff that isn't always fun. Everyone should share in the work.

At the same time, I have a foolish hope that someday I'll find a company that gets IT and how to manage people. I'm more than happy to come back to the fold if management can fix the employee situation.

The Burning Ends Annual Memorial Day Post of 2013

It's become sort of a tradition that I post on Memorial Day. So here we are in 2013 and the cynical bastard is at it again.

American Sniper

I've just finished reading American Sniper [Amazon.com] by Chris Kyle. It's Kyle's memoir about his life as a US Navy SEAL. As SEAL biographies go, it about par for the course with plenty of action and anecdotes about the grueling training program and missions in Iraq. It's told with the help of professional writers and they do a good job of capturing Kyle's personality. Contrary to what some grunts probably believe about any of the special forces teams and their PR engines, Kyle is very reverent to other services, paying respects and recognizing they were truly a bigger part of the conflict than a few sniper teams. He is repeatedly humble about his achievements always felt like he was there to help others.

Kyle would have probably been proud to refer to himself as a Texas redneck, but there is something admirable about the no-nonsense common sense and honesty in this country boy's words. Kyle isn't shy about sharing his opinions and in this day of political correctness, I appreciate that even if I didn't agree with him on everything. Every day, and on Memorial Day especially, we honor our veterans by respecting free speech and free thought.

As I read the book, I found it echoed many of the themes I cover here at The Burning Ends. Kyle ranted about how administrative overhead could interfere with getting the actual job done. He dealt with a SOX-like regulation of his own where he had to document each shooting and have witnesses, and how sometimes those rules of engagement meant he had to refrain from sniping a questionable contact that might have gone on to harm other people. But Kyle wasn't stupid, he understood why those rules were there and even conceded that the recordkeeping had a benefit of helping him track his work.

The Decision-Consequence Gap goes to War

The book also reminded me often of my last post, The Decision-Consequence Gap. Careerist officer types more concerned with looking good on paper would withhold Kyle and his teams from working the dangerous zones where they could be more effective at helping troops. By being able to report low or no casualties the officer would look good. This is analogous to IT leaders that do not empower their teams to understand, master, and improve processes, hardware and software that will ultimately improve the business. In sports, that approach is often referred to as "playing not to lose" rather than "playing to win". I'll give the officers in question a bit more slack than Kyle does though, as human life is a harder chip to play than user comfort and efficiency or a little more profit.

The Cost

But there is always a cost for managerial complacency and selfishness. In Iraq, was the cost that more line troops lost lives without the overwatch duties Kyle's sniper teams could have provided?

That's an important point about cost. I wrote earlier about free speech and free thought and I do believe that how we carry ourselves and cherish the benefits of the American way is a measure of respect for veterans (recall Saving Private Ryan's [Amazon.com] admonition at the end of the film: people died for you to have this life. Make it worth it.). But Memorial Day is really about the cost, not the benefit.

So to every veteran, thank you. Again. We can't say it enough.

The Difference

And for you idiots that think all things military are bad, well, too bad. I know, there are Iraqi and Afghan citizens that call US soldiers the terrorists, and if someone lost a child to collateral damage from a drone-fired Hellfire missile, I sympathize for a pain that I wish no one would ever have to endure. But that goes for the children of 911 too, and I don't take kindly to being called a terrorist because there is a difference between us even with the mistakes we've made.

Kyle mentions Mike Monsoor, a SEAL that dove on a grenade to save his fellow teammates. It's not a one-time thing. There's also the story of Specialist Ross McGinnis who did a similar thing in Iraq to save others. THAT is the difference. For all our faults, Americans at large revere life. You don't hear stories about insurgents doing such things, they're too busy blowing up civilians. America reveres life enough to spend significantly more on soldier training and equipment. To spend significantly more on search and rescue operations. That is the difference; the enemy our military has been fighting for the last decade would do no such things even if they could. Kyle made no bones about calling the enemy plain evil, and as crude a description as it is, it is also the truth.

Chris Kyle, Rest in Peace

It is on a somber note that I close this post. I purchased the American Sniper as an eBook in early 2013. I later found that recently Kyle passed away [ABC News]. It was not from war, but from a tragic incident in which Kyle was trying to help another veteran. It's a terribly sad thing to happen after what Kyle endured in dealing with war and coping with the challenges of returning to civilian life and focusing on his family. But he tried to help others because he cared about life. And that's the difference that we should take from it.