My Memory of George Perez

George Perez died May 7, 2022. His story is amazing, and I hope not rare, just easily remembered by me because I was close to it. It's the classic scenario of a child of an immigrant family that succeeds almost entirely on the grounds of hard work. 

Perez was an artist. I was almost tempted to say "comics artist" because he primarily made his living illustrating comic books, but just as a truly raceless assessment of a person should be made on the grounds of the individual's achievements and contributions and not genetic heritage, an artist is an artist, regardless of the medium. Perez was a great one. Self-taught and a tireless worker, Perez illustrated hundreds of books over his career. He deservedly became a fan favorite due to his style's crisp detail and great layouts for action and large group scenes.

I was blessed to be a kid in the 1980's, when Perez and Marv Wolfman were helming DC's The New Teen Titans. The title was a hit and really DC's only competition to Marvel's phenomenal line-up headed by the X-Men. It was the perfect title for teen readers as it had the superhero staples, but also discussed issues maturing teens dealt with like growing up, becoming independent, relationships, and dealing with parents or parental figures. It was nice that I could buy those issues at retail price when they came out. The 80's were a time of high price volatility in comics speculation and being a popular title, Titans back issues could get expensive. 

Perez was a force for a long time, and continued to be popular as he worked on other titles, notably the Wonder Woman reboot, also with Wolfman. The pairing of Wolfman and Perez is a special one. It was the right combination of writer and artist. Their stories went beyond the rock'em sock'em roots of early superhero comics and elevated comics with mature themes and welcome forays into non-violent paths. At least a couple of their Wonder Woman issues involved no fighting at all. 

The work the pair did certainly inspired me and I'll be forever grateful for their efforts. 

It was probably in the mid-1980s that I got to meet Perez briefly at a convention. I went with my best friend at the time, and we sat near the front row of the hall where Perez was speaking. I remember being in awe at learning Perez was self-taught and perhaps that was the beginning of my recognition that we should limit the gravity we impart to the superficialities of a college degree. 

I'll always remember Perez's response to the audience asking him who his favorite superhero was. "It was Iron Man. When I was a kid growing up in the Bronx, wearing a suit of armor made sense!"

Godspeed, Perez, to you and your surviving family. You affected more lives than you know. 

Collecting Sunsets

Speaking of Funerals...

Oh boy, here's an interesting if macabre topic. It's actually quite appropriate coming after my last post about letting go. 

As I've gotten older, I've started seeing end-of-life events. Not talking about humans here, we all have to deal with that, but about systems and businesses. It's something that as a young IT person you don't think you'll ever see, but then Father Time comes by and whacks you in the nuts with a baseball bat.

I don't think I've ever been involved in an end-of-life for any systems I helped start, or where I came in for some work during the middle. But I have now seen a few in just the last couple years where I've gotten involved in a system during its twilight years. 

There are a couple of distinctions to make about these first. 

• One type of end-of-life is witnessing the actual death of an application or system. It's dying and going away for good.
• Another is seeing systems die but only in a specific implementation. The software may still be alive elsewhere but is standing down in an installation, perhaps as a part of a company closing.

Total Death

The first end-of-life I saw was a true death of a system I supported for an oil and gas company. It was a home-grown system built around 1997. It handled some business for this company until 2022. Twenty-five years is a respectable life for a corporate system, often some of the world's worst written and most poorly designed systems. This app had an interesting life; the software was heavily used and had a team of 15 people at one time supporting and enhancing it. About 2012 the company split and the software was still being used for about half its functions. I was part of a team brought in to keep it and a sister system (a third party software package) alive during the transition. We were probably expected to just be there a couple years and before the client migrated to a new system. 

This system was proprietary, old, clumsy, and ugly. No one wanted to work on this bitch. The full-timers at the company wanted nothing to do with it, and everyone on our team preferred to work with the sister system since it was an industry standard application that was to continue use long after this system was replaced. Well, guess who got assigned to take care of it? Yours truly. But I'm the kind of consultant that takes care of the customer, even if it's a career limiting move, because that's what I'm being paid for. I hope, probably in vain, that the qualities of loyalty and reliability will be as useful in garnering future opportunities as simply having the latest acronyms on the resume. So I took it on and became a one-man show, handling all the support and even improving the software some. From a team of fifteen to a party of one...not sure if the client appreciated that but I absolutely provided value.

Real life threw a curve ball at this company and a downturn in the energy industry around 2014 caused the company to turtle and lay people off left and right. Everyone on our team got terminated except for the one person that knew how to keep alive the system that was still managing business and producing invoices. Again, yours truly. And it wasn't just contractors being let go, I outlasted the dozens of full-timers that didn't want any part of this system. This taught me a useful lesson that's not a tautology, but can be a useful tactical move: sometimes it's good to be good at the job no one wants.

Things picked up by 2016 and some of my team was brought back. But between budget cuts and project delays, I last another six years, for a total of eight, with this client. The sister system was upgraded and  picked up most of the business the proprietary system handled. The proprietary system was finally feeling death creep closer. I'm actually released from service in late 2020, so I miss the actual system shutdown that happens in 2022. But I was close enough to capture the essence of it; activity slows to a crawl, there are no enhancements to be worked on anymore, and even support dwindles. Yes, frankly it got a little boring at the very end, though it was certainly less stressful than when things were busy. 

The client mothballs the system for archival purposes and ceases using it for daily work. The system is then buried under layers of time. Many people that worked with this system would say "good riddance," as do I, but also I feel a little sadness. It's like all the creativity and work of a generation just becomes a tiny footnote in the company's history...and maybe not even that. Just gone. If systems had souls, I would say to this one, you helped some people in your life. That's meaningful even if it's forgotten.

Partial Death

The second story is of a system's sunset in a specific implementation. On one contract, I'm brought in to join a team that's supporting the systems for a company that's just been sold to a holding company and everyone knows it's going to be sold again. We don't know when so we carry on as professionals and do a good job knowing it could end at any time. It does, about two years after I sign on. The first year I was there was pretty busy; we had ongoing business and multiple systems to manage, and we also did some upgrades and generally carried on like a regular business. In the middle of the second year, the sale of the company is announced. Any enhancement or upgrade projects are instantly canceled and we shift into support-only and business transition modes. 

In this case, the products don't die; the two key systems we supported are both third party products that as far as I know are thriving in the market. At this company though, they're going to cease functioning. Not so sad for the applications, but more for the people side of things as the group is going to be broken up. The staff splits between a group that transitions to the new owner, a group that voluntarily leaves, and the ones that would have liked to stay but are asked to leave. 

No matter how it happens, it is as DeMarco and Lister said (and I paraphrase slightly in brackets): "Whenever a system goes live [or dies], someone gains power and someone loses power."

Closure

Why did I choose to write about this today? I think it's because it's September 29, 2023, the last day of Netflix's DVD operations, known as DVD.com. The service pioneered internet-based service companies twenty-five years ago and was also a leader in the transition to streaming and taking a hand in content production. The DVD side of the business naturally dwindled as streaming's convenience looked like a tall and handsome alternative to a short, ugly, and older option, and humans are pretty predictable in those circumstances. A few of us holdouts still appreciated the qualities that the physical media service offered, but even we had to know it wasn't going to last forever. 

Netflix of course, is still a Wall Street darling and is doing well, but I can't help but feel a little nostalgic as I remember the DVD service's halcyon days and how much fun it was to add movies to a queue and have them appear in the mailbox a few days later, and at a much more reasonable price than what the buster on the block was charging. 

In my opinion, the drawdown of the Netflix DVD business is a stellar model of how to sunset something. 

• First, communications went out months in advance to notify customers this was coming. 
• Then, the website was updated with a FAQ pre-emptively addressing concerns customers would have. 
• The service continued at a high level of quality and professionalism until the end. 
• On the last day, DVD.com even sent out parting gifts. The company sent a few extra disks from the titles still in the customers' queues, and clearly explained that any disks customers had in their possession past September 29 could be kept if desired. 
• Finally, the company executed a graceful departure, issuing thanks to customers for their patronage. 

This is an end that was sad, but also well executed and one that didn't leave behind unsatisfactory mysteries or a sense of being ripped off, as can happen when a magazine shuts down and subscribers are left with unfulfilled issues and no refund. This was an end with a comfortable closure, so much better than the sloppiness of America's departures from Vietnam or Afghanistan. 

Here's a capture from the site as it was on the last day. 

Actually, thank you, DVD.com for doing it right and showing us a successful exit implementation. 

Sometimes You have to Let Go

Americans live in a disposable society. We've progressed to this point gradually and over decades, and I don't know exactly when it happened, but we've been here now for a while. If you've read any of my previous posts about repairing appliances and computers, you know I'm not fond of it. I believe a quality product should be built to provide service for a reasonable amount of time, and commonly worn parts should be available cheaply so do-it-yourselfers can extend the item's life. 

That's not what we have now, and we haven't had it for a while. Today it's easier to throw something away, even in still good condition, and replace it. And this attitude is not confined to hardware. It's happening with relationships and marriages too. The hardware I can sort of understand. Corporations running the manufacturing want to keep making more money and they'll make more selling new printers and cars than they will individual parts. Don't get me started on the shenanigans Hewlett Packard is pulling these days with printer ink and toner. The relationships thing is also problematic but more about humans becoming increasingly focused on self and laziness rather than the particulars of an endless need to increase profit. 

That brings me to this post. It's about my old HP Laserjet 5P laser printer. I bought it new around 1996 and it served me well through nearly thirty years of light but regular use. The only thing I ever had to worry about was replacing toner. About ten years ago, the toner cartridges started becoming more scarce. HP had long moved on from the old 5P and wasn't really into making supplies for printers that were older than any of the executives that still worked there and were in small enough numbers that it wasn't profitable.  

I got by on third-party toner for a while and occasionally still picked up one of the last intact OEM cartridges when I could find one on sale. Through Windows 98, Windows 7, Windows 10, and the birth of my children and their passage to college, my 5P soldiered on. It was a great printer, built to last and quite versatile. It had a straight-through paper path so it rarely jammed, and it could handle thicker materials like label sheets and card stock. It also had a small manual feed tray that popped down from the front so I could do a one-off print if I needed to do it on special paper. Those don't sound like special features when compared to modern printers, but remember this 5P came out during much earlier days of PC computing. It also can print wirelessly, although through and old IRDA port instead of Wi-Fi. 

I haven't needed to print large volumes of documents in the last decade or so, but would still occasionally print important contracts. The 5P also proved quite useful in recent years as I printed shipping labels for all the stuff I sold for charity on eBay. 

Then, last year, the printer started to falter. The print would become increasingly faint and it got progressively worse. I scoured printer support forums looking for ideas on how to solve this problem, but after cleaning the transfer roller and trying yet another new toner cartridge, it persisted. I'd invested somewhere around $80 in toner cartridges in the last year trying to fix this but I'm now pretty sure toner isn't the issue. I see that I can buy a new fuser, transfer roller, or even laser scanner assembly for about $20 each but if I guess wrong and end up having to get all three, now that's another $60 I'll be out and there's no guarantee that'll solve it. I've reached the point of diminishing returns, where any more money spent on repair attempts will cost more than a new printer.

In addition, the 5P has just plain started to degrade. The four rubber feet on the bottom liquified into goopy messes years ago. The plastic used in the housing and parts on the interior has become brittle. A few years ago when I opened the back access panel a couple tabs broke off, although the printer continued to work. Whenever I open the power port panel or serial port panel, another piece breaks. Every time I pick the printer up now I worry something else will snap off. 

Is it time to cut my losses? I can get a new monochrome laser that'll be smaller, lighter, support wireless printing, and perform duplex printing. It'll run less than $150 (the 5P cost $800) and should give me several years of use, and toner cartridges should be easier to find. I still need to print occasionally, and right now my workaround is to use sneakernet to print on my second printer from a USB drive. I can probably live with the workaround, but it is a hassle. 

It sometimes feels like admitting defeat. But nothing lasts forever, and twenty-seven years is a darn good run. It's time to accept moving on. HP Laserjet 5P, you were a legend. This isn't a divorce, it's a funeral. It's not "get out," it's "goodbye and thank you."

The Mysterious Ryzen Reboot Bug

There's an old saying about boat fanatics. "The happiest days of a boat owner's life are the day he gets a boat and the day he sells the boat."

It's funny because it's true. A boat gives you access to happy (we hope) times on the water where you can swim, fish, sunbathe, and race. But the asset is often expensive to purchase and like most things, can come with a high cost of ongoing ownership. 

You can't say the same thing about a PC. For PC owners, it should be, "The happiest days for a PC owner are the day he gets a new PC and the day he gets an even newer PC." That's because a PC isn't a boat...it's a commodity and you expect it to just work and only get better as you climb up the tree of PC evolution. 

The Problem

Up to about a year ago I'd run on a Dell 5675 Ryzen 7 1700X PC that was mostly great. For the first three years it was solid and a good performer. It could run games fine and kicked Windows in the ass. Then, in late 2020 when I added a second monitor for work-from-home (WFH) the system would start to randomly crash after a short period of idling. It didn't yield an error, it didn't point to a specific app, it just would reboot. I could not figure out what the source was. Because it started happening after I went to dual monitors, I thought it was a video driver issue. I had the latest drivers for the GPU (an nVidia 1060) but the problem persisted.

I was not Alone

Online, I found that others had the issue too. There were a myriad of ideas, but no one could really pin it down. Some found it was related to power management, some theorized a Windows issue, some felt it was due to a faulty CPU, others said it was a faulty motherboard, some said it was the BIOS. 

And sometimes the testimonials contradicted each other. One person would say it was only on one particular chip, but several people with different Ryzens were seeing the issue. Some said it was due to a bad motherboard maker but the issue was happening across different motherboard brands. 

The Long Road of Diagnostics (and Trial and Error)

In my personal experience, it might have been some weird combination of all those things. Updating drivers did not solve the issue. 

Reinstalling the Operating System

I got a temporary reprieve by refreshing my Windows 10 install. It seemed to settle the system down and for a month or two, it ran great again. The problem returned and I knew that going through the hassle of refreshing the OS install would only bring temporary relief. I knew there had to be an explanation but I could not figure it out.

Checking the RAM

I then looked at the memory chip. It didn't appear to be the issue. It was a single 8GB DIMM chip. I bought another 8GB DIMM to swap it out with. No change, the reboots would still happen with either chip. The one good thing about getting the extra 8GB stick was now I had effectively upgraded the PC RAM to 16GB. 

The CPU?

I wasn't doing any fancy overclocking and the PC had been running fine for years, so I didn't think it was the CPU. But I made sure to update the drivers and the BIOS and verify both it and the RAM chips were seated firmly. Nothing changed.

The GPU

Because the issue seemed to start when I went to dual monitors and was running Citrix to run a remote desktop from work, I wondered if it could have been the video card. But again I'd made sure the drivers were updated and I wasn't doing any overclocking. I tried changing the video cables. I tried going back to a single monitor by turning off and disconnecting one of the monitors. No luck.

Time to Make a Change

Through most of 2021 and the first half of 2022 I limped along with the issue. Sometimes it would be kind to me and not trigger for two weeks at a time. Sometimes I'd do the Windows refresh trick and buy a couple months. The issue was like the taxman, it would always come back for me. 

Here's a quick bit of info about my personal PC history. Since about 2008 or so, I've only upgraded my primary desktop PC about every five years. You'd think that's not often enough given that I play games too. PCs are good enough now that I don't feel the need to change every two years like I did in the 80's and 90's. 

As we rolled past the midpoint of 2022 I figured one way to solve the reboot issue was to replace the PC. It was about time anyway as the Dell was getting to be five years old. So I began researching what I was going to do. 

I've tended to alternate between building my own PC and buying a prebuilt. I enjoy building PCs but sometimes a mass PC producer can hit a pretty good price point and save you the time of having to order parts and put a PC together. I compared some builds at HP and Dell against a parts list I built at PCPartPicker.com. Because GPUs were ridiculously expensive at the time, I decided to go with the AMD Ryzen 7 5700G APU. I'd had great luck back in 2012 when I built a PC using the AMD A10 chip. It was a great chip; no, I couldn't run games at the highest settings, but it was more than good enough to run everything I needed, was great at handling Windows, and it would save me the cost, space, and power of having to use a discrete GPU.

As the time came to buy the components, the price of the 5700G spiked. So I looked at the Ryzen 5 version of that chip, the 5600G. It was nearly $100 cheaper and although it had two fewer CPU cores, its performance compared favorably to my existing Ryzen 7 1700X. I pulled the trigger on all the parts and looked forward to an infusion of upgrades coming in the form of a more modern motherboard, CPU, NVMe SSD, and a healthy 32GB of RAM. 

It Lives!

I put the 5600G system together and it was great. Performance was good, the motherboard came with a BIOS version already capable of handling the CPU on the first boot, and the RAM and NVMe drive are fast and I wasn't getting any random reboots. 

It DIES!

What the fuck. After two blissful weeks the new system started to spontaneously reboot as the old system had. I was so frustrated. Again I started the scouring of the web for testimonials from others with the same issue. And there were definitely others that had the same issue. But as before, everyone had a different solution.

Several found out it was a power issue and that the Ryzens have trouble with power fluctuations as the PC tries to conserve or feed more juice as the PC's needs change. Some fixed it by switching the Windows power profile from balanced to high power. Others went into the BIOS and turned off overclocking or slightly increased CPU core voltage. Some increased voltage to the RAM.

It's so damned frustrating to not know exactly what it was. But it really did seem like there was something to the power angle, especially since this seems to happen when the system idles and not when it's active. I can't believe the Ryzen APU itself was broken, but perhaps gradual degradation or change in the chip after running with power for two weeks led to an instability and required an adjustment to fix it.

I tried many of the suggestions online and as of right now, I've not had any crashes after turning off these three power-related settings in the BIOS:

• Disabled the RAM "Power Down Mode"
• Disabled "Precision Boost Overdrive"
• Disabled something else I'll have to look up.

All of the victims of this had different solutions, but shared a common suffering. It was painstakingly time consuming to endure so many reboots, changes of settings, futile research, and absolute exasperation. At least a few just returned the CPU and tried a different one. I'm on the cusp of going to Intel if I can't get this working. 

Back to Basics

I repeated a step with the 5600G system that worked with the 1700X system. I did a Windows reset/reinstall. And it worked. The system was stable again. And has been since. I'm really happy with the system.

Looking back, the problem started after I'd updated a bunch of AMD drivers for video and CPU. What this tells me is that one of the driver updates caused the issue. So this time around, I only installed driver updates as Windows recommended them, rather than preemptively going to the AMD site and downloading the latest. It's disappointing that I can't trust the latest drivers, but clearly there's something off in either the driver or the site recommendations on which drivers to use because the system is rock solid now. There's something in the video or CPU drivers that must be yielding perhaps a memory conflict. 

There was a time when I always liked to go get the latest drivers. Now, for this PC at least, I'm adopting the old adage: "If it isn't broke, don't fix it." 

Annual Memorial Day Post: 2023

Here we go. I'm going to potentially doom myself and write about an emotionally charged topic. Perhaps this post will be censored or taken down or I'll be burned alive in social media court, but remember it's just my opinion.

Talking About White Privilege

When I first heard the phrase "white privilege" I didn't like the concept and I didn't believe in it. I have always tended to agree with the argument that the answer to racism is not more racism, and outward finger pointing is just about the worst way to solve a problem. The finger should start by pointing inward, and honestly making sure you're not the one full of shit. 

I didn't like the sense that bandying around the catchphrase "white privilege" seemed to be accusing others of racism, when they may not have been racist. The false accusations and declarations of guilt without a trial so common in media today are signs of our civilization's regression to something barbaric. The smart are losing ground to the stupid and it's depressing. 

But I took a harder look at the concept of "white privilege" and although the extremists at either ends of the spectrum might disagree with my reasoning, I've come to an acceptance of it by recognizing it as an extension of a concept I learned about in management studies. 

The Halo Effect

In behavioral science, the halo effect represents the tendency for certain physical traits to impart the benefit of the doubt to the possessor of the trait. An example is height. Height is often perceived as a leadership indicator, even if the tall person is a complete idiot and couldn't lead himself/herself down a one-way street. With time and experience, the truth of a person's capabilities becomes known, but fresh off the starting line, people with halo effect traits have an advantage.

The most common example I remembered from management classes where I read about the halo effect is beauty. Attractive people tended to get the benefit of the doubt on first impressions as trustworthy or competent. In a group of people all applying for the same position, the most attractive person had the edge in being given the opportunity. There's some biological explanation here too, I'm sure, as most would like to be around people considered attractive. 

And yes, lighter skin is something that could be considered a positive trait. We'd like to think that isn't the case. We'd like to think that everyone is judged on their character and competence and not by how tall they are or the color of their hair or skin. But biology tells us that isn't the case. Even the most aware, intelligent, and disciplined of us can't change the fact that we're human and can succumb to stereotypes and preconceived notions when developing a first impression. 

Demographics or Bias?

I have met people of all colors and can't draw any definitive association about racism to color. Yes, I've met white people that were rude and made racist comments, and I've met non-white people that have done it too. Conversely, I've met wonderful people of all colors. 

As I said before, this is why I didn't like the idea of racial privilege. But that was smart me trying to reach for the ideal. I tried to reason that a white person has an advantage in an environment where whites are the majority. It made sense to me that people fear the unfamiliar and are more comfortable with the familiar. In a predominantly white population, it wasn't intentionally hostile discrimination but simply familiarity that would favor people of similar traits. 

Then I heard that lighter skin is a halo effect trait and that even in populations that are non-white, the lighter skinned generally benefitted from it. There are a number of studies on this and I won't get into it here but this told me I was incorrect in assuming it was merely a matter of demographics (although I'm not ready to fully discount demographics as at least a factor in the equation). The halo effect, or reverse halo effect, could be active in any environment and like height or beauty, skin tone could have an effect, perhaps as a part of the perception of beauty.

Another superficiality that imparts the halo effect is the shape of eyes. I bring this into the discussion because in a totally different culture, you can see how a trait can have a similar effect. The Japanese graphic novels (manga) and animated cartoons (anime) are huge components of their culture's media consumption. In many of these stories, villains often have narrow or slanted eyes while the heroes often have circular eyes (and also often, lighter skin). 

The Treadmill

So, ultimately, I've sort of come around to the idea of white privilege as a part of the halo effect. This does not mean that the people engaging in awarding or benefitting from the halo effect are racists. You can make the argument that it is a form of discrimination and that it has negative effects, especially when exercised by gatekeepers of opportunity, and that's an acceptable argument. We cannot however, assume that all people engage it in or that they are hostile. I'm living proof that you can be non-white and make a sustainable living by being employable and offering some value to an employer. 

What finally convinced me to accept the concept of white privilege? Corny though it might sound, it was a video game analogy. In many games you may control a character that earns experience points and then gains levels, which then allow you to improve your character's abilities. This system of advancement is often referred to as a "level treadmill" as you keep grinding on the treadmill to earn new levels. 

I came to think of it this way: On the level treadmill of life, the easiest difficulty setting is White Male. I'm ok with this analogy because it made sense to me. It's not saying the person on the treadmill is evil or racist which is good because they probably aren't. It's simply acknowledging the halo effect. 

How do we solve this problem? I think it isn't solvable. It's like an incurable but manageable disease. Human beings are imperfect, that's part of what makes us human. It is through education and communication that we improve and learn and develop the wisdom to work against our base instincts, but you can never remove the base instincts. And it is very natural for humans to be wary when faced with something different or unfamiliar. But we can manage it better by installing better leaders and better teachers. That's probably one of the best ways to attack the issue. Unfortunately, we have problems in politics and business and especially law where questionable people achieve leadership positions. And we have problems in education where administrators are doing their best to make the teaching profession unattractive...but those are discussions for different blog posts. 

I'd Rather be Here than Not

I need to tie this post into Memorial Day, so here it is. You may not like the level treadmill, which can also symbolize the Sisyphean life of working for retirement, but I'm darn glad to have it compared to some of the alternatives (like slavery). And we have it because someone died to keep us from some of those worse alternatives. Gratitude as always for our veterans.

The Abominable 1222 Posi-Temp cartridge

Have you ever had to replace a broken cartridge valve for a shower/tub? I've done it now twice in my lifetime, and may yet have to do it again a few times before I go. 

The cartridge is the Moen 1222 Posi-Temp device, and in some ways it's ingenious. It allows a single handle fixture to control a shower/tub's water. You pull on the handle to open the water flow, turn the handle left or right to modify the temperature. They're found in tubs, showers, and tub/shower combinations.

The cartridge is small and easy to install, but after sitting in a wall and getting used daily for ten years they eventually wear out and break. Commonly, you'll find that you suddenly cannot close the faucet and the flow won't stop. Or maybe it starts leaking, or perhaps you can't control the temperature anymore. Anyway, it breaks and you'll have to replace it. Even if you get a complimentary replacement cartridge from Moen (they will send you one free if you're the original owner of the home), you'll still have to undergo the labor to perform the replacement work. You could hire a plumber, of course, but I'd recommend doing it yourself if you are moderately handy. 

Removing the shower handle is not difficult. The problem comes when you have to remove the old 1222 cartridge. The things tend to get stuck pretty bad in the brass valve body. The first time I had to do one of these, I pulled on the old cartridge and it wouldn't budge. I had to use a puller that allowed me to slowly use the power of torque to gradually pull the cartridge out, and I saw the cause of the resistance. There are rubber oval grommets on the left and right sides of the cartridge and as I was pulling them out, one had folded over on itself and created considerable friction. Fortunately, I was able to work through it and remove the cartridge. Putting the replacement cartridge in is super easy. 

But recently I had to do another cartridge extraction and this one was worse. I used the same tool I had before, but after reaching the maximum distance it could pull, the cartridge was still stuck. I used a pair of channel lock pliers to wrestle with the cartridge and managed to get it a little further out but it was really stuck good and wouldn't budge. I increased the force and the housing cracked and the cartridge broke in half. Now I had half the cartridge, a mass of plastic and metal, still stuck in the brass housing. Yuck. And you have to be careful here because if you damage the brass valve body you've just turned a $50 job into a $500 job. 

WHY? Why in 100 plus years of plumbing technology evolution have we not figured out a better way to manage this? Now I have to go caveman on this thing to finish what should be an easy job.

All right, it is what it is. With a mix of screwdriver, drill, chisel, and needle nose pliers, I break apart the innards of the remaining assembly and finally get it clear of the brass. During the extraction I could see the rubber grommets had sort of gotten stuck in the left and right passageways on the sides of the brass (where the hot and cold water come from). I wonder if that caused some of the resistance. 

I am not the only one dealing with this problem. There are dozens of videos on the internet of others dealing with this. Common approaches to resolving it are:

• Use a hair dryer or heating tool to heat the area and help loosen the cartridge.
• If the main valve rod breaks off the cartridge, use a drill and a tap to get a grip on the inside of the cartridge and then pull it out.

Both of those didn't apply to my situation. In my case, I had to literally tear the cartridge apart bit by bit. I did get the idea to use the drill from one person's video. You can CAREFULLY drill the side wall of the cartridge housing and this can loosen it.

The best solution for handy work though is to have the right tools. Here are two that can potentially make this job easier:

• The Lakeview Puller [LakeviewPuller.com]. This baby is $175 and probably worth it, if you're a plumber that gets called on to do this task regularly. It's similar to the puller I have now but it's longer and has a slot up the top and bottom to accommodate the rectangular panel at the base of the cartridge stem (the one identifying hot/cold with the letters "HC"). This tool is designed to attach to a drill, so once you're hooked in correctly, the drill will make fast work of the extraction, even if the grommets are causing issues. It's a great tool, but probably too expensive for someone like me that just does this once every 7 years. 
• Another tool option is the FlowRite ONA Puller [Amazon.com]. It's $75 at Amazon and has attachments that'll handle cartridges similarly to the Lakeview Puller, but with elbow grease rather than a drill. 

If the next cartridge I have to deal with proves to be a total bastard, I may go ahead and buy one of the above tools. A competent person can do anything, and a competent person with the right tools can do anything more efficiently.

The Job of Factual Verification

The title of this entry is from Jay McInerney's Bright Lights Big City. I read it a long time ago and don't even remember exactly whether the phrase was serious (did the main character work in a fact checking department) or sarcastic (perhaps a treatise on mundane work). But I thought about it when remembering an article in Vietnam magazine about the battle of Ben Tre. 

The article noted that at the end of the battle there was a US officer that quoted as saying "It became necessary to destroy the town in order to save it.," a quote captured by reporter Peter Arnett that would become one of the most famous lines from Vietnam and that would symbolize the futility of the war.

The Vietnam article says the quote is commonly incorrectly attributed to a US Army officer, claiming it was actually a US Air Force officer (although this was a long time ago and I'll have to find that issue and reread it - I reserve the right to correct this blog entry if I have misremembered the article). I have often thought of this claim and didn't have resources to challenge it. But recently I was doing some spelunking around on the internet and this quote came up again and sent me on an odd rabbit trail.

The Ben Tre quote and its origins appear to have been the subject of much controversy. There are conflicting accounts of who said it, there are some who accuse Arnett of misquoting or even inventing it, and still others who claim to have witnessed it and directly attribute it to a name (Arnett did not reveal the name of his source, saying only that he'd spoken with several officers about the battle).

The most convincing appears to be a captain of the US Army Corps of Engineers, one Michael D. Miller. He's got his own website [nhe.net] where he posts some remembrances of his time with the engineers in Vietnam. He's proud of some of his team's accomplishments, such as building schools and hospitals for the Vietnamese (and he should be proud). 

Miller was stationed in a town called Rach Kein, 25 miles northwest of Ben Tre. One of the pages on the site is dedicated to Ben Tre [nhe.net]. He did not participate in the battle at Ben Tre, but after the battle he attended a press briefing, and Miller's testimonial is that US Army Major Peter Booris was the person who said the famous quote. It's not quite clear if this was in fact a press briefing at Ben Tre or at Rach Kein, but if we are to believe Miller's account of the origin of "It became necessary to destroy the town in order to save it," then we have to figure it was in Ben Tre, where Arnett would have been. 

Still more questions arise as we keep digging. A 2018 article at Bloomberg.com has writer Stephen L. Carter pondering the etymology of the quote and ultimately finding that its gist existed far before Arnett's piece appeared. Carter notes several examples of the pyrrhic sentiment having appeared in WWII, the Civil War, and earlier. His earliest reckoning being from a 1908 Supreme Court case where Justice Edward White denied a modification to a law that he feared would change its meaning.

In Vietnam: A Visual Encyclopedia by Philip Gutzman, the entry for Ben Tre (pages 73-74) specifically states, "The quote was later revealed to have been made up by an AP journalist to add more drama to his report." This book is a big tome filled with interesting information but does not provide any attribution for its claim.

Another interpretation of the quote's origin is from Ralph Keyes. Admirably, Carter quite fairly includes mention of this in his article (although he quite unfairly [and twice] declares he's taking Arnett's word as unchallenged truth [in the absence of evidence we sometimes default to character, and given this I too would tend to accept Arnett's word, but that's me, a random idiot on the web...if you're a journalist, you have to go with what you have, not what you wish]). You can also read some of Keyes's argument here [aphelis.net]. Keyes brings an army major into the discussion, but not Peter Booris, rather one Phil Cannella. Cannella does not recall having said that exact quote but believes he expressed to Arnett, "It was a shame the town was destroyed," and considers it possible Arnett may have misquoted or embellished the statement. Arnett insists he got the quote right, and why wouldn't he to his deathbed and beyond? It made him fucking famous.

The Keyes piece also notes factual inaccuracies in the quote. Ben Tre was not completely destroyed as the quote indicates, and Ben Tre is too large to be a "town" and was a provincial capital. 

So, who said it? An army major or air force major, and which one? And were they misquoted? Did Arnett relay it exactly, or perhaps paraphrased with influence from a meme that started in 1908? 

The left wants to say the quote is perfect, and it certainly is a great sounding one even if fabricated. The right would say it was neither exact or accurate. The truth is I'm left with the same answer I started with: I DON'T KNOW. And that's the same answer anyone who wasn't there that's being honest has to give too.

I cannot offer a firsthand witness account. I wasn't there. Only two firsthand accounts exist that I know of: Arnett's and Michael Miller's. Miller seems to have a good case, if we can verify his army service and perhaps confirm the press briefing he attended indeed included Arnett. There could be other firsthand accounts, but as far as I know Arnett has no intention of naming the officer he quoted, although I'm not sure why that's sensitive. We're not giving up CIA spies here. Even if we had a dozen firsthand accounts, who's to say they aren't all bullshit? 

McInerney ends up being right both ways. Factual verification is important, challenging, mundane, and sometimes hopeless.