Mark Atwood

I predicted the new AWS EC2 "Spot Instance" pricing model a year and a half ago

me@mark.atwood.name — Sun, 20 Dec 2009 19:25:47 GMT

Way back in 2008-05-30, in this post, I said the following:

But then I realized that the better way would be to do it as a continuous dutch auction. I would specify the most per instance-hour I was willing to pay, and EC2 works out the lowest price across the entire universe of currently outstanding bids that will completely fill the capacity available for these things. As more people come into the system and bid for capacity, my instance-hour price I am paying can rise or fall based on everyone else's bids, and I could have my instances shut down if the current market clearing price rises higher than my set max bid.

There is competition to EC2 coming online soon. Eventually, someone is going to try this charging model.

If I was to build an EC2-like system, it's the charging model I would use. It would most perfectly capture and monetize my value to my users, giving me the largest possible income for the value I am providing, which I can use to build more capacity. It would also give me a much better and smoother signal to how much my users like my system, other than "there is more capacity than is being used, I overbuilt" or "I'm oversubscribed, and at 100% utilization, I need to build out more".

As long as *anyone* bids into the auction, for more than the equipment capital and power cost of running their jobs, and yet doesn't get any cycles, because someone richer has outbid them, that signals to me, the grid operator, that it will be worth investing in more grid.

And now Amazon AWS EC2 has done exactly this very thing, calling it "Spot Instances", in parallel with their existing "per instance hour" pricing model, which basically puts a maximum cap on the auction price.

Am I the only person to publicly predict this?

Playing with QR Codes

me@mark.atwood.name — Sun, 20 Dec 2009 07:50:52 GMT

On our way to Santarchy

me@mark.atwood.name — Wed, 16 Dec 2009 18:48:24 GMT

Santas, originally uploaded by FallenPegasus.

Pictures taken of me by Cunning Minx at the Steampunk Ball last weekend

me@mark.atwood.name — Sat, 12 Dec 2009 02:34:58 GMT

PHP accelerator cache that uses memcached?

me@mark.atwood.name — Wed, 09 Dec 2009 18:38:42 GMT

Last year, when I was doing MySQL Professional Services, I encountered a client that was already using memcached. Something they said they were doing was they were caching the compiled bytecode of their PHP code in their memcached, which was a big win because they ran a large fleet of identical PHP based application servers. As soon as any one server encountered a given new piece of PHP, it would compile it and cache it, and immediately all the other app servers could use the same cached compiled bytecode, rather than repeat that work. They had recently changed to this approach, from caching the compiled bytecode on the disk of each app server.

I thought that was really neat, and kept digging elsewhere into their performance and scaling issues.

I had just assumed that this was some open source project, a modification or module to an existing PHP bytecode compiler / cacher / accelerator.

Except, it seems to not be. I've spend a couple of days now googling and reading up on the various "PHP accelerators", and they all appear to cache to disk or cache to local shared memory, but I can't find a reference anywhere to coupling one with memcached.

Am I just missing something, is my google-fu failing me, is this something this shop had written from scratch?

Do any of my readers know?

Work: Gear6 Memcached for the Cloud

me@mark.atwood.name — Tue, 08 Dec 2009 18:21:08 GMT

When I first started working with Gear6, I pushed for bundling a version of our memcache appliance as an AWS EC2 AMI. We had already done much of the work by making a version available as trial as a VM image and our current effort on a "Universal Distro", which can run on any qualified piece of server hardware.

Because of that existing effort, and because of the nimbleness and skill of my coworkers here at Gear6, we were able to move from proposal to first release in only a few weeks.

We are releasing our first version of the Gear6 Web Cache for the Cloud today of Amazon EC2, at ami-2411f34d for 32 bit, and ami-2611f34f for 64 bit. The 32 bit version is "free", you only pay Amazon's EC2 charge, and the 64 bit version (which can cache much more, return results faster, and handle more client connections) is linked to Amazon DevPay, so you will pay some money to Gear6. But it works out that the "gigabyte hour" cost for the new "high memory" EC2 types is actually less than the cheaper smaller "free" 32 bit size.

Jeff Barr at Amazon AWS just blogged about it. The press releases and tech press articles happen today.

It's been a learning experience for me. I got to deal more with the tech press, and learned more about how publicity and press releases work.

I also got to learn more and hard about Gear6's internal development and build processes. And I broke the build, several times, on the integration day. I need to get better at testing.

But I'm excited, I hope this goes well.

Thoughts on Maurice Clemmons, Mike Huckabee, Willie Horton and Mike Dukakis

me@mark.atwood.name — Tue, 01 Dec 2009 18:16:01 GMT

While I am no fan at all of Mike Huckabee or his ilk, I can't help but notice that many of the people calling for his head and/or chuckling at the damage that this has done to his political prospects are the same peoples who thought that the Willie Horton - Mike Dukakis affair was unfair.

They are also increasing the political pressure towards even more stupidly harsh sentencing and more executions, despite those things being against their own principles and ideals, for the sake of their own blind partisanship.

If a former governor is politically punished years after the fact for following the advice of the pardons and parol board, and the opinion of the sentencing judge, in deciding to issue pardons, especially to a minor who committed a non-violent crime, then soon enough, no governor will be willing to take the risk to issue pardons, except to wealthy well-connected powerful friends.

But the baying voices don't give a shit about such damage to our polity or our society, just so long as they get to pull out their partisan long knives, and take down a politician they don't like.

Feh on you all.

notes on the days

me@mark.atwood.name — Sat, 28 Nov 2009 05:42:15 GMT

In Utah, staying at my parents. Wasted most of the day sitting in the SEA airport, because I missed the gate change of my flight. My parents now have THREE cats. Jaysee is a little black skittish thing that technically belongs to some irresponsible neighbors, but actaully lives in the workshop. I had wondered why my dad was suddenly intent on insulating that building, after N years of letting it be cold...

My mom still uses the ReplayTV, even tho it no longer can change channels to catch OTA broadcasts, what with the end of NTSC transmission. She just uses it to pause whatever show she's watching.

Network television hasn't gotten any better.

Utah is still the weird little bubble. It's not the same weird little bubble it was 20some years, but it's still a weird little bubble.

Dad broke his foot yesterday morning, doing stuff out in the workshop getting ready for my arrival. 4th metatarsal. He was remarkably lucky, of the 3 ways it could have broken, he picked the one that is 8 weeks in a boot cast, instead of one of the ways that would send him to an ortho surgeon. He would have been even better off if he had gone to the clinic right away, instead of a few hours later. Sigh, my parents....

I got AT&T to give me the unlock code for my Sierra Wireless 885 wireless modem. But I can't get it to connect to T-Mobile. Got to just LOVE how Apple evicerated all the useful logging and detailed configuration files of PPP. To make it "more user friendly" and "less confusing", I'm sure. Mac's are great, they Just Work. Until they don't, and then there is no way to make them work.

Thanksgiving Day went rather well.

Wednesday afternoon I completely forgot an appointment at MassageFreek. I got deeply involved in stuff I was working on. Crud. And I had been looking forward to that appointment for a week.

My life is more interesting than it sounds, but a great deal of it goes into my personal journal now, not into my lj.

We will see what tomorrow brings.

Thought on touristy postcards

me@mark.atwood.name — Fri, 06 Nov 2009 19:13:03 GMT

Places that sell themed and/or local touristy postcards, such as the San Francisco Exploratorium, should also sell thematically appropriate stamps of the correct denomination, and have an outbound mailbox.

Sometimes, people just gotta be told what to do.

me@mark.atwood.name — Mon, 02 Nov 2009 06:00:13 GMT

Once again, there has been an upset little kid on the flight. This time, far in the back behind me. Not sure what she is upset over, but she's pretty clearly saying F*** You!.

I am committing presenteeism in the first degree, by being on this flight and by going to ApacheCon. I caught a flu last Friday while in Atlanta. By Saturday morning I had chills, hot flashes, stuffed sinuses, raw sore throat, and the real capstone was that all my joints ached. And I couldn't just sit in bed or in the bathtub, some of the furniture in the house has been swapped around, and it lead to me needing to move a bunch of my own stuff, and help one of my housemates make a run to the dump. So while everyone else was carving pumpkins, I was withdrawn, grumpy, and curt.

Speaking of curt, I had an odd experience at the end of my flight back from Atlanta. The flight had left late, and there were a handful of passengers on board who needed to catch a connecting flight to Anchorage. The crew announced as we were taxi'ing in that that other flight was holding and delaying at the next gate for us, and could everyone stay seated so the Anchorage bound people could get off fast. So, we pull up to the gate, and the there is the bing sound and the crosscheck alert. And, of course, EVERYONE stands up, gets in the aisle, and starts opening the overheads.

I am annoyed. So I say, "HEY! ARE YOU GOING TO ANCHORAGE?!" My voice rung out, filling the cabin, louder and harsher than I've heard it in my own ears in a long time. And there was silence for a moment, and then everyone sat back down, and a handful of people rushed down the aisle and caught their flight.

Sometimes, people just gotta be told what to do.

More musings

me@mark.atwood.name — Thu, 29 Oct 2009 13:09:23 GMT

(written yesterday, Oct 29, while in the air)

On my way to Atlanta for No:SQL(East). We really need a better name than "NoSQL" for the emerging implementations of scalable key/value stores and caches, but nobody yet has come up with one that has stuck. Which is maybe a good thing, since a "good" name constrains wide thinking (which is both a good thing and a trap), and right now this space of development is still exploring what "works" for APIs, usability, implementation, performance, and scalability. We dont want it nailed down yet.

There are, of course, a whole array of existing and soon-to-exist vendors who are commercialize some approach (fair disclaimer: I work for one of them), and then start pouring marketing muscle into nailing down people's thought patterns towards thinking of Product X or Approach Y as "THE solution to your problem". That will come, soon, for good and ill. I don't think anyone has jumped the gun just yet.

This is not the most comfortable flight I've been on. I'm in the back, I can smell the lav, the seat is hard, and in the row in front of me there is a wiggly little boy and also in infant who expresses the slightlest bit of frustration, bordom, tiredness, or being poked at by said wiggly little boy by howling. (She just feel asleep, thank prime. Hopefully for the rest of the flight.) This happens sometimes when travelling, and has to be lived with, sort of like the occational dead run though ORD. Crap, she just woke back up, mainly because her brother got bored and poked at her to see what would happen.

I am still undecided about my growing hair. It's been a year now since the last major cut. Fortunately, I dont seem to be suseptible to the "frizzy fringe of split ends" that many longer haired male geeks are. The length is now to where it touches my below my neck, a sensation I'm not yet used to. I can tie it back, barely, but I don't yet have the habit of stocking hair ties. I had a couple of them knocking around my room that had accumulated over the years, but I've since now lost them all. I finally figured out where they get stocked in the local supermarkets, but I went to each of them, and they were all sold out. What, was there some kind of recent super demand for hair ties?

SteamCon I, this past weekend, was fun. I really enjoyed looking at all the clothes and costumes. People should dress up more, with more colors, patterns, and things that swirl. We have the social and technological infrastructure to make attractive and comfortable clothes for relatively cheap, and yet most people dress so poorly.

Of course, I am one to talk, often being head to toe in black. My original and continuing reasons for doing so are complex and sometimes hard to articulate. There are elements of wanting to stand out, wanting to disappear, not wanting to have to worry about matching colors, having components that are easily replaced as they wear out, and having less stuff.

I did get a new vest and shirt and hat for the con, and picked up some jewelry while I was there, and they will get worn again.

I have taken to working from Metrix Create Space on Broadway when I am working, instead of a local cafe. They have tables I can spread a workspace out on, they have power outlets, they have ripping fast and dependable internet, interesting people come in to hack on interesting things and having interesting conversations

And something that is really nice is that they keep the background music low, and play classical chamber music and/or jazz. I know that some people claim they can work only when the music is cranked. I'm not one of them. And I really don't like the inclination that many places have, as described in the They Might Be Giants song "Man, It's Loud In Here!", to turn every place of social commerce (cafe, restaurant, bar, gym, clothing store, etc) into a place of loud music. Earbuds were invented for a reason.

I noticed today someone had muted all the TVs at SeaTac, at least in the terminal I was in this morning. They were instead running the closed captioning. I strongly Approve, and hope this is a trend that is spreading.

I tweaked my right arm at Burning Man, probably while tying down a piece of webbing while a windstorm was trying to tear off the back of the shade structure. It has been healing, slowly. But I've discovered the hard way to avoid both climbing and doing pushups, both of those retweak it. I had sierrafaye dig into it during my appointment with her yesterday, and that was a high intensity sensation that helped a great deal. I kick myself a little that I didn't think to have her dig into it earlier.

Some thoughts. On dressing up, on tagging faces, and on cliques

me@mark.atwood.name — Sat, 24 Oct 2009 19:56:35 GMT

Steam-Con has been fun, even just purely from an eye-candy point of view. More and more, I become a fan of dressing up, and of the people around me dressing up. Everyone looks good in good fun clothes.

The People in Photos feature that Flickr just rolled out has been much fun to play with. I've spent some time adding this new metadata to mine and many other photos. Some people have marked their photostream to not allow it, and some people have asked to not be identified. That request gets honored, but it's pretty much pointless. It won't be too many more years before automatic recognizers just get run over all photo and video sources, both live and historical. You will be tagged and identified, the only question is do you want access to your metadata as well. It will exist anyway. (But it's not terribly useful to point that out to people who think that by asking, it won't happen...)

I had an odd exchange a few days ago with a friend of a friend, which I am still musing over. It ended with me saying "it sounds to me like you maintain your relationship with a GROUP of people, instead of a set of individual relationships with individual people." (The middle part of the discussion was about my serious discomfort with observing that something looked like a self-selecting closed set "are you cool enough to be one of us" "cool kids" clique.) Anyway, her response was "that's the only way I can maintain a relationship with as many people at once as I want to have".

I can't tell if this is just a world-view difference between us, or a failure of communication and understanding, or something I've been "doing wrong" all these years. Maybe it's just that I don't much care for the "cool kids clique" attitude. But then, every handful of years, I get to learn the hard and painful way, that trust isn't transitive, that friendship isn't transitive, and that even fellowship isn't transitive.

SteamCon Portrait

me@mark.atwood.name — Sat, 24 Oct 2009 03:48:43 GMT

SteamCon Portrait, originally uploaded by FallenPegasus.

Finding and fixing bugs in libmemcached

me@mark.atwood.name — Tue, 20 Oct 2009 16:39:43 GMT

In Trond Norbye's blog entry Testing libmemcached on EC2, he refers to "Someone pinged me yesterday about a problem he was seeing when he tried to run the test suite on Jaunty Ubuntu.".

I am that "someone".

I ended up filing two bugs on launchpad against libmemcached 456080 and 456084, and just submitted a branch that fixes the second one.

Telephone Problems

me@mark.atwood.name — Mon, 19 Oct 2009 06:14:01 GMT

I think that T-Mobile has screwed up my telephone number port.

I can call people, and the CID is the correct number, but when they call me, they get a "number not in service" error.

I will be on the line with them tomorrow to get it fixed.

I called a couple of people last night, really hoping for a call back, and got nothing. This may be way...

Me on the beach at Waikiki, final sunset

me@mark.atwood.name — Sat, 17 Oct 2009 18:29:41 GMT

Mark wading watching, originally uploaded by peachiekissies.

My prediction about high fashion:

me@mark.atwood.name — Tue, 13 Oct 2009 07:32:09 GMT

In this post, my friend elfs comments about Karl Lagerfeld's reaction against the blowback against anorexic runway models.

I responded in agreement, with this comment:

I think I've told you my prediction that reasonably soon, the fashion design industry will eliminate the human models entirely, and instead have robotic coathangers with legs walking the runway. At which point it the last vestige of it being about wearing clothes will be severed, and it will quickly evolve into a variety of kinetic sculpture with textiles.

Then the rest of the world can completely ignore, instead of mostly ignore, the "high fashion" "designers", and can instead just wear fun clothes that we like wearing.

Randoms and random randoms

me@mark.atwood.name — Tue, 13 Oct 2009 03:23:08 GMT

It's been a month since I've been back from Burning Man. There is a lot I got there, and a lot I've taken away, mostly on the social processing front. A problem is, instead of getting a whole "something", I think instead I just got part of it. Which is maybe what is supposed to happen. Instead of being given a whole thing outright, I have some new or different threads to chew on.

The new job has been rewarding, it magnifies some of my favorite strengths, but also magnfies some of my least favorite weaknesses.

I've not been posting much. And I don't want to be one of those people who most of my posts have been that I've not been posting much.

I have been travelling more, and having some good experiences while travelling. Right now I'm in Hawaii. Not for any reason other than a friend asked me to accompany her to Hawaii for a week.

I'm using my wireless EDGE modem, until this data plan runs out on the 18th. AT&T has significantly improved their network in Honolulu from from when I was here last..

I'm not being so impressed with T-Mobile. I can't get wireless data inside the SeaTac airport. Fail.

I'm about to go out walking to see my friends at the Kapahulu SBC.

More later.

Don't worry about "Who am I?" and instead consider "What should I do?".

me@mark.atwood.name — Mon, 05 Oct 2009 22:19:02 GMT

There’s plenty I don’t understand about myself, but nothing nags. Paradoxically, the deeper I got into neuropsychology the less interested I became in the details of my own inner workings. I’m not sure why. It certainly is not because I arrived at any great insight or understanding. I still experience the almost visceral sense of puzzlement over matters of brain, mind and selfhood that first drew me to the field. What happened, I think, was a shift – let’s imagine a neural switch somewhere in the frontolimbic circuitry - from one preoccupying question, What am I? to another, What should I do? It left me less inclined to bother about self-understanding than to consider the value of things, moral and aesthetic. How best to live? But here’s a nagging thought: might those two preoccupying questions turn out to be one and the same, like the evening star and the morning star? -- Paul Broks

link

Twitter still uses the 3rd Party Password antipattern ...

me@mark.atwood.name — Sat, 03 Oct 2009 08:09:31 GMT

... to get one's contact list from Google.

!!!

Bad, Twitter, bad!

You know better.

A really odd way to build a new model of economic action...

me@mark.atwood.name — Wed, 30 Sep 2009 05:15:36 GMT

http://www.creditbloggers.com/2009/09/using-brain-scans-to-beat-the-free-rider-problem.html

As soon as fMRIs are super cheap, here is one way they can change things:

A research team at the California Institute of Technology (Caltech) recently conducted anexperiment in which a group of volunteers were offered an abstract public good. The price for the public good was fixed, but each group member would benefit differently from it. So how do you determine how much each person should pay? In this experiment, the researchers scanned the volunteers' brains using functional magnetic resonance imaging as the volunteers stated how much they were willing to pay. The researchers analyzed the fMRI images to obtain information about how much the person really valued having the public good, and then compared it with how much they were willing to pay for it.

The kicker was that volunteers whose measured value closely matched the amount they were willing to pay were charged less than people whose fMRI value was higher than the amount they were willing to pay. In other words, the folks who lied about how much the public good was worth to them were penalized by having to pay more than the people who told the truth about how much they valued the public good. As Caltech graduate student Ian Krajbich, who co-wrote the paper about the experiment for the online edition of the journalScience, put it, "The rules of the experiment are such that if you tell the truth your expected tax will never exceed your benefit from the good."

It didn't take many iterations for the volunteers to learn that honesty was the best policy, at least in this laboratory. The researchers said that once volunteers understood the penalty for lying, they told the truth 98 percent of the time.

As far as I'm concerned, the age-old free rider problem has been solved. Now all I need to do is drag my free riding neighbors into the nearest brain scanning center and make them pay up.

New Job. Director of Community Development at Gear6

me@mark.atwood.name — Mon, 21 Sep 2009 16:23:31 GMT

Many people noticed my tweet just over a week ago that I tendered my resignation at Sun Microsystems. Many people asked me "what next?". Here is my What Next:

Today is the first day of my new job at Gear6, as the Director of Community Development.

In part, this means that I will be doing for Gear6 what Jay and then Dups used to do for MySQL.

I'm the face of my company into the Memcached / Gearman / Drizzle / etc communities, and just as importantly, I'm the face of those open source communities into my company.

I was very positively impressed with the people at Gear6 when we were interviewing each other. I truly believe that they want to do The Right Thing, both from an open source community member perspective, and from the perspective of the technology, their customers, and their investors.

Very soon, I'm going to be splitting my "blog existance" into several blogs for different aspects. My current existing LJ will remain for my personal stuff. There will be another one for professional / technical musing, and I will also create and manage one or more blogs just for Gear6's open source and technology topics.

A working "progress bar" for a huge ALTER TABLE

me@mark.atwood.name — Sun, 20 Sep 2009 21:17:03 GMT

My friend Gabriel came up with a working "progress bar" for ALTER TABLE. Until MySQL / Drizzle can do this "natively", this is a pretty neat trick:

http://gabrielcain.com/blog/2009/08/05/mysql-alter-table-and-how-to-observe-progress/

Senior citizen's tourist shuttle driving around Burning Man...

me@mark.atwood.name — Thu, 17 Sep 2009 21:55:36 GMT

I actaully saw this!

I was surprised and a bit taken aback to see it, something that looked like a senior citizen's tourist shuttle driving around. It was something utterly alien and unexpected to see there.

http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2009/09/05/MNUN19INQL.DTL

I actually figured it was some kind of art. But now I know it was something even stranger: it really was a senior citizen's tourist shuttle driving around.

The record labels do not just "not get the internet". They do not understand their own business: ma

me@mark.atwood.name — Thu, 17 Sep 2009 17:24:00 GMT

Yesterday, while walking from the Online Cafe (where I can print and fax), to Kinkos (where i can fedex), I walked past the Everyday Music on Broadway.

I walked in for a moment, walked up and down the aisles, looked at the layout, at the staff. It was all very interesting, and the staff seemed very engaged. But I didn't buy anything. Because looking at CD cover art tells me nothing about whether I would like the music inside or not. And using iTunes or Amazon MP3 tells me in a moment with a 30 second clip what something sounds like, and a click later I own the piece, and am enjoying the full thing.

According to Gizmodo, the big music labels are now demanding that they get paid for the performance of promo clips.

!!!

This is sort of like having to pay a ticket to get to look at movie preview.

This is not the record labels "not getting the internet". This is the record labels not actually understanding their own business, of marketing, advertising, and branding!