Feedback during product development

We understand the importance of incorporating real feedback while developing a product to satisfy some market need. Last post also talks about not releasing in-progress product to a large customer base.

But, how does feedback mechanisms change as product evolves?

New products

I asked this question to couple of product managers, and their observation is to gather feedback at least 4-6 times during every product release for initial few cycles. And allow time to incorporate feedback back into product. If revenue goals are tied to quarter endings, time the product release slightly earlier so that any last minute glitches do not slip the release to next quarter.

Another important observation is that before demonstrating intermediate product for feedback, do a quality check to ensure release constraints and acceptance criteria are fulfilled, or any previously fixed issues do not reappear. And this check must not take more than 10% of calendar time during every feedback cycle.

Carefully select strategic stakeholders for feedback, and a small percentage of customer base as beta testers.

Finally, after last feedback has been gathered for a release, only issues must be fixed. Nothing new.

Seasoned products

Dynamics change for mature products. Keeping existing customers satisfied and recurring revenue becomes more prominent, and so does change in feedback mechanisms, quantity and quality. Product receives feedback from virtually every customer, and a larger employee count is required to sort out and categorize feedback, and maintain interactive dialog with customers.

As incorporating new features slowly takes back seat with every release of product, ironing out issues becomes more prominent. List of release constraints and acceptance criteria becomes longer. As number of fixed issues increases, regression testing become more expensive. Product engineering has to be more careful while introducing new features.

Majority of feedback now comes as a regular stream, rather than as discrete blocks collected during demonstrations.

But, the objective of product manager remains same. Inject newly discovered feedback into product release at a constant and predictable rate. A maximum period should be decided that a customer will have to wait to get the product release, once the issue has been selected for resolution. Keeping a constant rhythm and predictable dates for releases, allows customers to plan for upgrades fairly accurately. Again, keep some error margin so that minor slips do not negatively impact revenue goals.

Service- or subscription based consumption model also allows customers to enjoy advantages of upgrades almost immediately in most situations. Although, it might be a good idea to offer a roll-back option to use older version of product, should they choose so.

Marketing campaign: Wait, we are not ready yet

During initial launches of a product, it might be more prudent to invest on product usability before big marketing and PR activities.

It is even more advisable if the initial audience for the product is large. Everybody checks out your product, and it's not good yet. These people will be permanently convinced of inadequacies, even if you improve it drastically later.”

Many cash-rich companies, whether Microsoft or VC-backed startups, run the risk of launching to a bigger audience than they really should have, getting millions of people to be thoroughly unimpressed by version 1.0 and never bothering to come back again to see if 2.0 might have gotten it right.

Lesson: Release early, release often, but only to selected audience. Invest in marketing and PR activities, only when there is minimum critical mass of usability. Select customers and users who will stand beside, unless you seriously mess up!

The effect is explained as Marimba Phenomenon.

What is simplicity?

It is impossible to fuck it up :)

I need a car, not an airplane

I am sick of seeing products with too many features, to the extent I get confued.

Should good products have less features? Not necessarily! However, it is important that a new features in a product should be added only when

• when consumers demand for it, and/or
• when customers are ready to consume it

As a user, it is very embarrassing to look at a product and wonder what the hell does it do! Especially, if there are so many options and controls on the interface. And a complex-to-use interface generally means that the product is meant for more mature market than the users' current skill level.

Nothing wrong with that, but let me get mature with usage of product. Let me get used to simple things, and then I will request for more power and fine-grained options.

If a product is intended for broader market (from beginner to power user), figure out a way to expose only enough features to match users' expectation level and gradually expose them to more powerful features, as they become more proficient and demanding.

And yeah, as you add new features, also keep a tab on features which are no longer relevant, and remove them. It will simplify product design, and keep costs down.

[Thought provoked after reading a blog entry from Laura Klein.]

Another incident to prove importance of acceptance criteria

McAfee had a goof-up.

As soon as anti-virus was updated (and generally auto-updated), an important system file was quarantined. Windows would shutdown every 60 seconds, and USB would not work.

Now, it is impossible to accept that this was not detected during the final QA, when checklist of all acceptance criteria for the product release was being ticked. Assuming, there was a checklist, that is!

Lesson to learn: Product Managers, be ready with your constraints and acceptance criteria for a product release, and work through it with the engineering team. Follow "stop the line" principle from lean techniques.

Too many choices confuse customers

Give me something that will solve my problem, and I'll buy it.

"And, if you want to solve this problem too, you can add on ...."

"But, if you are not interested in , then you don't need to buy this...."

"And, if you are not sure, you should go for this..."

...

...

And, I end up with 10 different options. Overwhelmed. Confused.

Yes, I like options. But, only up to a count of 3.

More than that, I assume you are not confident in what you are selling.

Are you ready to listen?

Yes, social media is important. Interactivity is important.. blah, blah, blah..

More important: Are you ready for it? As soon as open the gates, there will be all sorts of conversations happening. Are you ready to deal with them? Act upon them? Show everyone - you care.

If not, you may just fall flat on your face, as you try to keep pace. Have you calculated the cost of not responding once you open your interactivity channels? Is your business process geared up well?

And yes, the quality of your groups does matter. Its important to know whom you are talking to? Understand the market reputation of customer, and the impact they can make on market. Even a small customer might make a big impact in market.

EDIT: another interesting post on same subject matter "Don't listen to your market"