While at Target, I had the opportunity to sit in on a user study on a kiosk application we were building to allow customers to load coupons onto "smart" credit cards to be used when they check out.

Subjects were grabbed at random from the flagship store and offered a modest gift card to participate in the study.  Each subject was brought into a small room with a kiosk in the center and a large two-way mirror on one wall.  

A facilitator would begin by warmly thanking them for participating and guiding them verbally through a pre-scripted set of actions.

"You can load electronic coupons onto this card, try to use the kiosk to load a couple onto it...", while handing them a prototype card.

The person would take the card, peer at it oddly, and gamely try sticking it into...the printer slot, look embarrassed, then find the correct opening.  The facilitator would be careful to look placidly unhelpful no matter what the subject did.  On-screen, they'd touch the "Load Coupons" text label next to the big button...wait a moment...touch it again...wait a moment...then finally touch the button.  

On the other side of the mirror, half a dozen people sat in rapture.  A row of three usability high-priests sat hunched over microphones immediately in front of the mirror.  They'd whisper intently, "the subject has tried to insert the card into the wrong slot" and "the subject is having trouble finding the coupon button", while a dozen video screens showed the action from various angles.

It was fascinating, and slightly gut wrenching, to watch a real person flail at the thing we'd worked very hard on for months.  Your first instinct is to blame the subject for "not being computer savvy", but eventually it sinks in that you have to do better.

The study resulted in a dryly written report that lived in a massive binder that weighed 15 pounds.  An imposing and technical-jargon filled document that, ironically, was itself far less usable then the application it was studying.  

I'd guess that I only ever read about 10% of it, and yet, the experience dramatically altered my perspective about products and the importance of user experience.  Products are only as good as their interfaces, and letting a real customer play with it can tell you a great deal about how to make that interface better.  Not by asking them for feedback, but by simply observing them and watching your application fail them, sometimes in the most basic of ways.