At one company I worked at, one of the problems it didn't have was IT.

When someone was hired, by the time they got to their new desk, there was a computer on it with the correct image on it, their desk phone worked, their email worked, the calendaring and scheduling worked, and all necessary passwords and ACLs were configured. The internal ethernet networks all worked, were fast, and were properly isolated from each other. The wall ports were all correctly labeled, and there where the right kinds of wall ports in each cubical and conference room. The presentation projectors and conference room speaker phones all worked. The printers all worked, printed cleanly, were kept stocked, and were consistently named. The internet connections were fast and well managed. Internal and external security incidents were quickly recognized and dealt with. Broken machines were immediately replaced with working and newly imaged replacements. If someone accidentally deleted a file, getting it back from backup typically took less than an hour. Software updates were announced ahead of time, and usually happened without issue.

The IT staff did not seem noticeably bitter, angry, harried, or otherwise suffering from the emotional costs traditionally endemic to that job role. In fact, they were almost invisible in their skill and competence.

So, of course, came the day when the senior executives said "the carpets are just naturally clean all the time, we don't need all these janitors!". IT was "reorganized" into a smaller staff of younger and much less experienced (and probably cheaper) people.

Of course, it all went to shit. New employees would go a week before they had machines, phones, passwords, and ACLs. Printers ran out of paper, projectors ran out of lightbulbs, servers ran out of storage, networks got misconfigured, and so forth. The total time lost and wasted across the whole company was most certainly greater than the savings of laying off the expensive and skilled IT staff.

This is not to say that the reorganized IT staff were stupid or lazy. They worked very hard and ran themselves ragged trying to keep up with the cycle of operations, while trying to skill themselves up in their "spare time" and with a slashed training budget.

The lessons I learned from this experience speak for themselves.

What lessons that may have been learned by any of the other people involved, especially the executives who made these decisions, I cannot say.

This entry was originally posted at http://fallenpegasus.dreamwidth.org/849512.html. Please comment there using OpenID.

Tags: bofh, it, stories
Current Location: Capitol Hill neighborhood, Seattle WA

I was working on a highly constrained consumer electronics device, a little "satellite device" that spoke to the main device over a CATV RF coax cable and also received commands from an IR remote control. My code was failing in bizarre ways. I adopted an extremely paranoid defensive programming stance, filling my code with asserts and doing paranoid cross checking of all inputs. This didn't make the device work. Instead it consistently didn't work, instead of inconsistently, because the cross checks and asserts would usually (but not always) trip before it would crash. It also started to run out of memory because of the all the paranoia code I had added.

I asked for the source code for the driver for the IR receiver, and for the driver for the CATV RF digital transceiver, and for the peer code that was driving the cable digital that ran on the main device.

The driver for the CATF RF digital transceiver was handed to me the first time I asked. And by "handed to me" I mean that I was pointed to where it was sitting in the source repo.


The business partner / hardware supplier who was supplying the IR glue and drivers just , after giving me a runaround, finally just flat out refused, citing trade secrets, confidentiality, secret sauce, and similar bullshit.

So, I finally "stole" the source code with a disassembler. And found the sources of many of my problems. It was complete shit. "Unexpected" input from the silicon would cause wild random pointer writes. And random sunlight on the receiver optics would cause it. "Expected" input of undefined remote commands wasn't much better, generating and handing back blocks of garbage with incorrect block length headers.

I ended up writing, nearly from scratch, a replacement IR receiver driver.


The peer device driver code was written by a developer in a different group in my same company. I finally got the P4 ACLs to read it after loudly escalating, over the objections of it's developer and his group manager. It was also complete shit. I cannot even begin to remember everything that was wrong with it, but I not only figured out may of the sources of my own pain, I also found a significant source of crash and lockup bugs that afflicted the main device.

I was not allowed to rewrite the peer code, as it was not in my remit. However, I was able to sneak in and check in a large number of asserts, using the excuse that they were "inline documentation".


On, and the device driver for the CATF RF digital transceiver? The source code I got for the asking, without a fight? When I reviewed it was easy to understand, efficient, elegant, and as far as I could tell, bug free.


In the end, I made my part work. It just took over two months instead of the original guesstimate of less than two weeks. This caused a schedule slip in the release of the satellite box. Which would have been a more serious problem, except…


Except there was also major schedule slip for the main box. A significant reason for that slip was because the peer code that I had filled will asserts was now crashing with assertion failures instead of emitting garbage. I was lucky that I was not more officially "blamed" for that. The reason why I wasn't, was mainly because the people who understood what I did understood the problem, and the executives who didn't understand what the problem was were also too clueless to blame anyone, let alone me.


My lesson learned from this experience is: if someone is refusing to show the source to suspect driver code, citing trade secrets, confidentiality, secret sauce, partnership agreements, or similar excuses, it's not because they are protecting their magic. It's because they have screwed up, and they are trying to hide it.

A second rule of thumb I have is: source control systems that don't allow any developers to check out and review any arbitrary source code file are expressions of moral failure. It is unethical for an engineer, designer, or other technologist to ever sign off on a project that has been mutilated by such a broken tool.

This entry was originally posted at http://fallenpegasus.dreamwidth.org/849188.html. Please comment there using OpenID.

Tags: open source, software, stories, work
Current Location: Newark Airport, NJ

It's really annoying to be so aware of a non-rational mental block in my own head, know it's origin, understand what it's trying to protect me from, know how and why it's counter-productive, and have that piece of non-rational pattern of thought also be the very thing that is stopping me from letting go of it.

This entry was originally posted at http://fallenpegasus.dreamwidth.org/848959.html. Please comment there using OpenID.

Current Location: Home, Capitol Hill, Seattle WA

I just went on a half-hour binge reading about pertussis, as a result of reading that WA state has an epidemic outbreak that the public health agencies are struggling to stop.

Reading people's first hand accounts of having it will cross your eyes in empathized pain. One person described that she has given birth, has suffered a compound fracture of a leg, has passed a kidney stone, and has had whopping cough. Only one of them made her wish for death. It's been described as feeling like an asthma attack while someone is punching you in the ribs. Oh, and most all cough medicines do pretty much nothing for it.

You, reading this. You. Right now, pick up your phone, and call your doctor's clinic. Ask them if you're up to date, and if you're not, go get your freaking DTaP shot. You can get one at Walgreen's for less than the cost of a high-end Starbucks drink. You owe it to yourself, you owe it to public health, you owe it to your friends and coworkers, and you owe it to every pregnant woman, every newborn, and every immunocompromised person you share this biosphere with.

This entry was originally posted at http://fallenpegasus.dreamwidth.org/848789.html. Please comment there using OpenID.

Current Location: Capitol Hill neighborhood, Seattle WA

If you found this personal blog from some link about my work on open source and open standards work, my work on OpenShift, on Drizzle, or my speaking at and participating at some technology conference, or so forth, you probably want my tech blog over at Machines Plus Minds.

This personal blog is entirely for stuff about my personal life, and if you don't know me personally, you will probably find it utterly uninteresting.

This entry was originally posted at http://fallenpegasus.dreamwidth.org/848424.html. Please comment there using OpenID.

Current Location: Home, Capitol Hill, Seattle WA
Current Music: (Joey Fehrenbach) Beltenebros

The daffodils are blooming again.

It's now been three years since I wrote Love Stinks, and a year since I wrote Love Stinks, Two Years Later

I read them, feel the scars again, and muse. Any fundamental change? Not really.


I do not retract a word of it. But my outlook has become more nuanced.

Some of that nuance is even more bitter, some of it more gentle.

I understand more. Understanding does bring peace. Or, at least, it moderates ineffective action.

Things that use to hurt all the time, now only have the occational pang.

I've gotten better at keeping my teeth together, and my tongue still.

I have gotten better at knowing how and how much to trust people.

Some people are selected out from my life.
Some people have drifted into my sphere.
Some people have drifted out.

With some, there has been rapprochement, and a friendship again.

Some old acquaintances, have become new friends.
Some old friends, have become new friends.

And there have been some pleasant surprises, and some new adventures.

I still don't trust "love", but friendship and affection are nice, and I try to enjoy what of it I have, from where and when it is offered.

This entry was originally posted at http://fallenpegasus.dreamwidth.org/848130.html. Please comment there using OpenID.

Tags: love
Current Mood: thoughtful

Comparing the crowd at Denver International Airport with my recent post about the crowd on the Metro bus in Seattle.

The racial mix is significantly more white. The people are generally taller and more stocky. There are a lot of people with tanned skin. Many of those tans are leathery. Faces, hands, arms, and necks, everywhere I look I see signs of significant long-term sun damage and other weathering, almost as bad as what I saw in Los Angeles. Their clothes are just as layered as in Seattle, but significantly more warm, and in darker colors. Lots of heavy wool full jackets, and lots of heavy weather coats. A lot of blue denim jeans. Lots of waterproof shoes and boots with high-traction soles, and showing a lot of water, salt, and snow damage. What "flair" there is, if any, is of a southwest and/or cowboy theme.

This entry was originally posted at http://fallenpegasus.dreamwidth.org/848038.html. Please comment there using OpenID.

Tags: airport, people watching, place
Current Location: Denver International Airport (DIA), Denver CO

My fellow passengers:

A large black woman, her hair short dreads. Wearing black t-shirt layered with green vest with jenin jacket. Brown socks with sandals. She's reading a technical manual.

An elderly woman with pixie haircut, sky blue northface coat, open, with two layers of thin sweaters, over a bloase. And a hand knit scarf. Yoga pants. Brightly colored knit socks with sandels. Her eyes look at nothing.

A muscular white man in his late 20s, tightly stretched blue t-shirt with ironic text, blue jeans with rolled and ironed cuffs, scuffed leather ankle boots.

And so on. About 2 dozen people, a mix of white, hispanic, asian, native, and black.

All wearing relaxed layers, mostly in subdoed earth tones, water resistant sport wear, comfortable shoes. Lots of dyed hair in subdued colors, crazy tights, nose rings on office workers, and hoodies. There is not a bit sunlight tanned skin in sight.

If I didn't already know I was in Seattle, inside the city proper, one look at this crowd would tell me where I was.

This entry was originally posted at http://fallenpegasus.dreamwidth.org/847866.html. Please comment there using OpenID.

Tags: seattle
Current Location: Capitol Hill neighborhood, Seattle WA
Current Music: (JES) Like A Waterfall (Flipside Ambient Remix)

It's getting to be that time of the year again, and I like sending cards for the holidays. So here is the deal: tell me your postal address, and you will probably get a card around the end of the year.

This entry was originally posted at http://fallenpegasus.dreamwidth.org/847449.html. Please comment there using OpenID.

Current Location: Home, Capitol Hill, Seattle WA

Today, while waiting in line for an ATM at my credit union, I got to witness two interesting customer service interactions.

The first customer was dealing with the aftermath of an all-to-common PayPal issue. Her account had been emptied via PayPal charges and/or chargebacks. The BECU agent said they see that sort of thing all the time. He had the information on hand to report a fraud to PayPal if her account had been compromised, or how to start fighting with PayPal to get her money back if PP themselves were grabbing her money. He also had a process in place to set up a "valve" account keep PayPal at arms length from one's money.

After she left, the next customer sat down. He was a trim white-haired guy, a prototypical small businessman. He was holding a big stack of envelopes branded with the blue Chase logo. The BECU agent asked him what he needed, his response was a blunt "I'm tired of these fuckers, what can you do for me?"

And then it came to be my turn at the ATM.

This entry was originally posted at http://fallenpegasus.dreamwidth.org/847356.html. Please comment there using OpenID.