Nested Lists

The Life of an Idea

2011-06-08T10:33:00.000-04:00

This morning I read a quote from George Orwell who is supposed to have said, "Any life, when viewed from the inside is simply a series of defeats."

Well, I'm not sure if that's true, but like all interesting quotes, it makes you sit up and pay attention. It does seem to apply quite nicely to the ideas I have though. A while ago I drew this "Krebs cycle" like diagram on the lifecyle of my ideas.

Better Late...or Never

2011-01-09T15:13:00.000-05:00

I've been putting off writing this for months now. Why? Because procrastination is not easy to overcome.

The idea came to me when I read James Surowiecki's book review in the New Yorker. It was published on October 11 and I must have read it around the first week of November, I think.

As I read the piece I recognized some of my own mental habits that make me procrastinate. They fall into two areas.

I. Uncertainty about achieving a result in a given time.

The result need not be a theorem -- it could be a diagram, a paragraph, a bit of conceptual progress. But when you begin, you don't know what's going to come out of the inquiry. And this uncertainty just doesn't sit well with my need for extreme productivity.

It’s uncomfortable grappling with unformed concepts, ones that are nebulous and unstructured and can be structured in many different ways. The same goes for arranging a constellation of well-formed ideas - how should they be ordered? How are they best related? The constant analysis, synthesis, re-analysis, and re-synthesis is scary. There's no way of knowing if this is a dead end or a productive line of inquiry.
This leads to needing to learn more about the concepts, but not being sure of what exactly to learn and how to proceed. The potential for wasting time is great and the urge to prevent this by not starting at all usually overcomes all motivation.
Not knowing how to proceed in concrete steps. This comes from not knowing how to break down the inquiry into manageable bits, and is closely related to the first bullet above.
Trying to hedge against lost productivity by having a preconceived notion of “time blocks” and not being able to fit the activity into an available time block. This stops me from starting a lot of things!

II. Uncertainty about the value of the task.

The task seems of low value compared to everything else I have to do. This is irrational - I do this even when the long-term value of the task I’m ignoring is high! It might be related to the “hyperbolic discounter” concept in cognitive psychology.
I'm too bored to write down what I've thought through -- it just feels unbearable to put pen to paper!
Can't get past the feeling that I deserve to be doing something else -- usually this comes from making artificial distinctions between work time and play time and then not doing something because it happens to be time for play, not work.

Only in Software Engineering

2010-10-13T11:34:00.000-04:00

I’m usually on trains late at night.

On one such train ride, I was reading Fred Brooks’ new book, The Design of Design, listening to Jeff Foxworthy, and periodically looking out the window (take that, ye who doubt my multi-tasking capabilities).

So, I give you, Only in Software Engineering…..

Only in software engineering would you define quality simply as meeting functional specifications. (Would you define the quality of your car that way?)

Only in software engineering would you measure everything *BUT* the product that you’re building – the thing that is going to generate value. (It’s like Usain Bolt measuring everything he does *except* his 100 meters time.)

Only in software engineering would you put together a diverse number of elements to make a single giant application and then pretend they seamlessly fit together.

Only in software engineering would you test the quality of the product after it’s already too late and impractical to do anything about it.

Only in software engineering would you collect reams of data about how fixing defects early save you gobs of money and then ignore it.

Only in software engineering would you insist that building software should be just like building a widget in a manufacturing production line even though you’ve never been able to do it this way.

Only in software engineering would you profess that repeatable process is sufficient for producing a high quality product.

Only in software engineering would you painstakingly catalog best practices and then ignore them.

Only in software engineering would you prove theorems about NP-hard problems and then claim that your test coverage plan is adequate.

Only in software engineering would you equate software quality with testing.

Only in software engineering would a software engineering book written in 1975 be just as relevant and correct today as it was 35 years ago. (I’m talking about Frederick Brooks’ classic, The Mythical Man Month.)

And from our Chief Scientist, Bill Curtis:

Only in software engineering would it be the case that the worse the initial specification, the more certain the estimate needs to be. (Curtis’ Conjecture on Estimative Dissonance – named for our Chief Scientist, Bill Curtis who first coined it in a webinar he did on Agile techniques.)

And finally, also from Bill Curtis,

Only in software engineering do the Indians always beat the cowboys.

By the way…

If your dad walks you to school …because you’re both in the same grade, you may be redneck.

That’s my favorite Jeff Foxworthy line.

C A S T Announces Software Psychoanalysis Service

2010-04-01T16:28:00.001-04:00

Happy April Fool's Day from CAST!

C A S T Announces Software Psychoanalysis Service

View more documents from CAST Inc..

Diffusion

2010-03-21T17:10:00.000-04:00

Recently, I noticed something about the way I think.

Suppose there's a way to measure how closely two ideas are related. I don't know of a way to do this offhand, but I can think of a way to construct this similarity metric. For the moment, assume we have it.

Then, a chain of thought would have a diffusion coefficient, just as a drop of ink in water does. The lower the diffusion coefficient, the tighter the relationship between the ideas. The higher the diffusion coefficient, the looser the relationship between the ideas.

Sometimes, the diffusion coefficient of my thoughts is high – I flit from one thought to another, sometimes even one task to another. Sometimes the diffusion coefficient of my thoughts is low – I can stay on a tight cluster of related thoughts for a long time.

As for the end product, it doesn’t seem to make much of a difference whether the process was a low or high diffusion coefficient one.

But here's what I noticed: my level of satisfaction seems to be much higher when the diffusion coefficient is high.

Juxtaposition

2010-03-20T13:41:00.000-04:00

Spent the day yesterday in Chicago at the IT Executives Club meeting in the Union League Building. Walked around with an old friend who gave me a quick architecture tour.

On the plane back I was reading Rishi Narang's book on quantitative trading. I had the pages open to a chapter titled "Alpha Models: How Quants Make Money". I happened to glance down at my seat mate's book, "The Healing Power of Crystals".

Thought 1: Never in the history of the universe have these books been open side by side! Can the difference be any greater?

This was very quickly followed by thought 2: Maybe they're not that different after all!

For Software Quality, Context Matters

2009-11-09T20:44:00.002-05:00

If you've been in any airport in the last few years, you've seen ads from HSBC - the global bank that prides itself on local knowledge.

The point is, context matters. The same outfit means opposite things depending on where you are or who you're with. The same thing applies to software quality. Quality is not just a local thing -- only when you have global knowledge can you act effectively on the local level.

Two of my colleagues, Olivier Bonsignour and Bill Curtis recently wrote an article explaining this at length. Here is a summary of their excellent article -- well worth checking out in full.

"Quality is not an intrinsic property of code: the exact same piece of code can be excellent in quality or highly dangerous depending on the context in which it operates. Analyzing the quality of modern applications in the context of the numerous interconnections with other code, databases, middleware, and APIs is monstrously complex. It can only be accomplished with automated software that analyzes the inner structure of all components and evaluates their interactions in the context of the entire business application."

They go on to show the quality problems that arise when context is ignored. Again, I'll summarize. You can find the detailed examples here.

Typical application quality problems are listed below to clarify the distinction between application and code quality. Performance testing alone is not sufficient to detect these application quality problems.

A. Bypassing the Architecture. Components in one tier of a multi-tier application are typically designed to access components in another tier only through an intermediate “traffic management” component. Bypassing this traffic management component will usually result in a cascade of problems.

B. Failure to Control Processing Volumes. Applications can behave erratically when they fail to control the amount of data or processing they allow. This problem is often caused by a failure to incorporate controls in each of several different architectural tiers.

C. Application Resource Imbalances. When database resources in a connection pool are mismatched with the number of request threads from an application, resource contention will block the threads until a resource becomes available, tying up CPU resources with the waiting threads and slowing application response times to a crawl.

D. Security Weaknesses. Applications are vulnerable to security attacks when they lack appropriate sanitization checks on user inputs in all relevant tiers of the application.

E. Lack of Defensive Mechanisms. Since the developers implementing one tier cannot anticipate every situation, they must implement defensive code that sustains the application’s performance in the face of stresses or failures affecting other tiers. Tiers that lack these defensive structures are fragile because they fail to protect themselves from problems in their interaction with other tiers.

Each of these application quality problems will result in unpredictable application performance, business disruption, data corruption, and make it difficult to alter the application in response to pressing business needs. Reliably detecting these problems requires an analysis of each application component in the context of the entire application as a whole – an evaluation of application rather than code quality.

Q&A on Software Quality

2009-11-01T13:02:00.000-05:00

A recent article from James Powell of Enterprise Systems magazine on software quality — it’s a Q&A format with yours truly providing the A.

The gist: Moving beyond software testing to understand and measure the structural quality of the entire software system is the key to preventing software glitches.

Check it out here: Improving Software Quality

Is Free a Four-Letter Word?

2009-10-15T00:13:00.000-04:00

We live in an amazing world! Shake a phone, get restaurant listings. Listen to the sound of a space craft landing on Saturn's moon Titan, without leaving your desk! Check the wind speed in Patagonia, from Peoria. See what Maru is up to this week. Find your rooftop on Google Earth. While you're at it, check out what your neighbor's house sold for (sorry, that's so 2008).

A couple of years ago, Nicholas Carr (of "Does IT Matter?" fame) observed that three things - bandwidth, storage, and processing power -- were simultaneously and rapidly declining in cost. Amazing things happen when this combination becomes virtually free.

Or "Free", as Chris Anderson puts it in his latest book. By now you're probably familiar with the dust kicked up by Malcolm Gladwell's review of "Free" in the New Yorker. Anderson's key idea is itself susceptible to being misconstrued, so there are plenty of straw-man rebuttals kicking around as well.

The idea of "Free" is a mesmerizing one. Here it is: "From the consumer's perspective, there is a huge difference between cheap and free. Give a product away, and it can go viral. Charge a single cent for it and you're in an entirely different business... The truth is that zero is one market and any other price is another." (I'm quoting Gladwell who in turn is quoting Anderson.)

Gladwell distinguishes four types of claims about Free.

A) Technological: Digital infrastructure is free

B) Psychological: People love free products

C) Transactional: Free means no barriers to entry -- anyone can create information, anyone can participate

D) Commercial: Free stuff can make you a lot of money

To these four, I add two more that seem relevant to the topics Anderson covers:

E) Teleological: information/knowledge "wants" to be free

F) Practical: The best way to create an information-based product is through a free exchange of ideas.

Gladwell argues that (A) is false (although YouTube is free for the user, it costs Google a lot of money to keep it running), and that claims (B) and (C) together undermine (D). Even if that's true, there's lots of room to maneuver. It's true that a low barrier to entry can result in products of low quality. But it isn't necessary - you could add a quality filter to improve the contributions. Or you could filter out certain contributions. You could argue that this amounts to a barrier to entry. But this conflates participation with contribution (I participated in the election, but I didn't contribute to Bush's win in 2004).

(E) and (F) are deeply embedded in the open source software movement (Richard Stallman and Eric Raymond make eloquent cases for them). I'm a fan of open source, but nothing I say here relies on the truth these claims.

"You can tell whether a man is clever by his answers. You can tell whether a man is wise by his questions." (Naguib Mahfouz).

In that spirit, I ask four questions -- the key questions to answer if you want to produce a Free product that also enables you to be make money (not by selling the product alone of course, but by selling something associated with that Free product).

Here are the four M's of Free.

1) The Market: What kind of information to aggregate?
2) The Mechanism: How best to aggregate this information (for free)?
3) The Multiplier: How to maintain or improve the value of this information?
4) The Business Model: How best to package and sell this information?

While the answers to these questions are difficult to rattle off, it's easier to say what the attributes of a good answer should be.

Attributes of the answer to question 1:

1a) Unique: Information you can't get anywhere else (Chowhound or Amazon reviews)
1b) Productivity Booster: Information product that turns something hard into something easy to do (Westlaw for case preparation)

Attributes of the answer to questions 2 and 3:

2a) Easy to Use (AOL CDs)
2b) Continually Increases in Value (Wikipedia)
2c) Versatile: Can be used to do more than one thing (Facebook, Linux)
2d) Powerful: Does something others can't do nearly as well (Bloomberg terminals, Linux)

As for question 4, there are three well known models that work with Free.

4a) Free product, charge for services (RedHat)
4b) Free product to begin; product is good enough to lock the user in; then charge for the product (Facebook and Gmail are good potential candidates)
4c) Free basic product; charge for advanced features

Which model you adopt will depend on the strength of your product attributes. If it's very easy to use, very versatile, and continually increases in value, then switching costs will be high and the lock-in business model will work. If the advanced features are unique, boost productivity, and are powerful, then charging for them will work.

There's nothing different about information- or knowledge-based products: they make (or lose) money just like regular products. But our amazing technological advances make possible some amazing knowledge-based products. How we create and capture value from them is something well worth thinking about.

How Gap Can Make Me a Repeat Customer

2009-10-03T23:46:00.000-04:00

I'm fascinated with irrationality.

There's willful irrationality where you don't bother to get the information you need, or don't use what you have, or ignore relevant information. At the extreme end of that scale is what the ancient Greeks called "akrasia" -- the disposition to act contrary to one's own considered judgment about what it is best to do (The Routledge Encyclopedia of Philosophy, Edward Craig, ed. 1:139). This is the stuff of tragedy, the rise and fall of empires, and the downfall of many a politician.

Then there's irrationality you can't help -- like the biases that Kahneman and Tversky so brilliantly catalogue. Even writers of advertising copy fall for advertisements!

Another type of irrationality stems from a failure to be instrumentally rational. You know the other side has a valid argument, but you just can't bear to act on it. You know you should walk away, but you don't. "It's the principle of it damnit!," you hear yourself thinking. At the extreme end of this scale is the willful failure of self defeat.

Today I had a bout of instrumental irrationality.

I live near Dupont Circle. There's a Gap store a few blocks down Connecticut Avenue, so I walked there to get a few things. After the clerk had rung up my items, he said, "Do you want to save $50 today? You can, if you apply for a Gap card!"

Now, I'm not crazy. I know $50 is a good sum of money. But I didn't do it. I'm not sure why I didn't.

It might be the hassle of filling out the form, just when I'm looking forward to getting out of the store. (It's certainly not about receiving spam, which doesn't bother me at all.)

It might be because I felt manipulated.

It might be that I was just irrational -- but as Dan Ariely would say, predictably so, because I never, ever sign up for these cards.

I don't know how many others are like me, but I was thinking about this incident as I walked home. If The Gap really wanted me to buy more from them, the clerk should simply have said, "Thanks so much for shopping here. We really value you as our customer and want to give you $50 back right away!"

What would they have lost? My name in a database, tens of emails that I would have deleted without reading, and snail mail fliers and catalogues that I would have just thrown out (I'm very predictable there as well).

What would they have gained? A repeat customer. I don't shop often, but you bet I would have walked down to that Gap again, or had a look at their web site first when I needed casual clothing.

Just giving me $50 without hassling me to fill in a form would have been the most effective marketing they could have done, with a customer like me. The marginal return of having my name in a database is miniscule in comparison.

So perhaps they are being instrumentally irrational too.

If they're so hell bent on getting my contact information, why not give me some warning? Put up some signs and let your customers know right when they walk into the store that you have big discounts in exchange for contact information. Have the forms on display and give people a safe space to fill out the forms (but don't have store employees hawking or cajoling shoppers to fill them out). I might have filled out a form then, perhaps.

Just because you have an offer that should be irresistible to a rational shopper doesn't mean the offer is irresistible. Plan for irrational shoppers.

Retailers including Gap already tap into this irrationality with the feng shui of merchandise display and placement, store layout, music, lighting, and other sensory factors.

Why not also plan for customer irrationality at the checkout counter?

Be Like Usain Bolt - Measure Output

2009-09-19T12:44:00.001-04:00

In a recent post I wrote about the craziness of just measuring the things you do to build something, without ever measuring the quality of the thing you build.

Imagine Michelangelo Buonarroti measuring the force and angle of his blows on stone without any sense of the beauty of the statue emerging from these blows?

Imagine Usain Bolt clocking his sleep time, his training time, his eating time, his Playstation time, but somehow failing to clock the time it takes him to run 100 meters!

Sure, that's crazy. But it's EXACTLY what happens in software development, enhancement, and maintenance projects.

Now, software projects do suffer from the Michelangelo problem: how do you define and measure the "goodness" of the software you produce?

Usain has it easy in this regard. How can we make the software product measurement problem less like Michelangelo's problem and more like Usain's?

Here's how you define and measure the quality of the software you produce.

The CAST Quality Model

For any given technology/language, CAST goes right down to the smallest unit of that language/technology. All metrics below are captured at that fundamental unit level and then rolled up all the way to the application level. The application-level metrics are also rolled up to the application portfolio level. An application may consist of many different languages and technologies -- that's a major strength of the CAST platform: it provides an end-to-end view of the entire system from database, to application logic, to middleware, to business logic, to the user interface.

There is extensive documentation on how each metric below is defined and how it is computed. But it's worth a short explanation here. CAST measures the degree to which an application satisfies a set of rules. At the foundational level, you can think of CAST as three things:

a set of highly sophisticated language parsers (to parse source code in 30+ technologies/languages),
a knowledge base of 900+ rules of software engineering (culled from the engineering specs of the technology, industry standards such as SEI, MITRE, ISO, etc., and field experience from our existing clients), and
a highly sophisticated comparator or rules engine that calculates the degree to which an application's components (and hence the application as a whole) complies with the 900+ rules of software engineering.

The language parsers and the knowledge base or rules evolve to keep pace with changing technologies, languages, and software engineering standards and practices. They rules are kept up to date by our engineering team.

Rules in the knowledge base are classified by technology/language. For example, there is a set of rules for Java and another distinct set of rules for C++. There is also one general set of rules that is independent of technology/language.

Each rule in the knowledge base is assigned a series of weights - each weight tracks that rule's contribution to a quality metric (weights can be zero). A complete assignment of rules to weights is called a Quality Model. The weights of the quality model can easily be tailored to particular production environments in which the systems operate.

Once the source code of the software system (multi-tier, multi-platform, multi-language) is parsed, the rules engine goes to work assessing the degree to which each basic component of the system complies with all rules that apply to it. Some rules will apply to sets of components; for example, how a set of UI objects calls a database object.

Quality measures are rolled up to two high level metrics: The Technical Quality Index (TQI) a gross measure of the quality of the software system, and 5 Health Factors:

Performance - Business Productivity
Security - Peace of Mind
Robustness - Responsiveness to Business Needs
Changeability - Speed to Market
Transferability - Rapid Knowledge Transfer Between Teams

So there you have it. A way to define and measure the quality of the software output you produce.

When it comes to software, be like Usain!

Anatomy of a Process

2009-09-08T20:39:00.001-04:00

When I think process, I find it useful to break it down.

When I hear process I want to know the following:

What’s the input?
What’s the output?
How repeatable is it?
What’s the frequency or cadence?
What impact does it have on the rest of the process chain?
How is it related to other processes?
What are the common variations and how frequently do they occur?
How are exceptions managed?
What are the escalation protocols?

what are the triggers?
how are the thresholds set?

What are the operational costs?
Who owns it?
Who administers it?
Who are the participants?

What is their organizational structure?
What are their incentives?

What are the success metrics?
What are the operational metrics?

Two Overlooked Drivers of Application Complexity

2009-09-07T18:32:00.000-04:00

There's a lot written about how the Applications portfolio has steadily become more complex. But there's very little attention paid to two significant drivers of this complexity.

One of the least talked about drivers of complexity in IT is the sourcing of the infrastructure to one vendor and the sourcing of applications on which they run to another vendor. This has disaster written all over it.

Typically, there's a many-to-many map between application components and the infrastructure components they run on. Good luck (a) tracking down the root cause of the problem, and (b) fixing it quickly.

The second overlooked driver of complexity is the shrinking Infrastructure footprint (due to a combination of data center consolidation and virtualization). Infrastructure shrinkage drives application consolidation:

more infrastructure consolidation => more application consolidation => more interfaces => more inter-application connections
more infrastructure consolidation => more physical distance between where an application is housed and where the global user base is located => application performance problems
more infrastructure consolidation => more users per system => application performance problems
Infrastructure is getting more consolidated while the environment in which applications are consumed is getting more fragmented. In particular, the degree of geographic distribution and the intensity of local customization is a lethal combination. Applications heads face two equally unpalatable consequences due to actions taken by one of their partners over which they had no say and no control. (and they are put in the position of reacting to what someone else has done over which they have no control): increase in application complexity, and a decrease in application performance (The Applications Death Spiral).

These drivers of complexity put a tremendous strain on the already frayed coordination between the Infrastructure and Applications functions.

The Pitfalls of Coordination between the Infrastructure and the Applications Function

Incentives are not aligned
Language is not aligned -- different metrics, different ways of measuring and capturing value
Each side has responsibility but no accountability
Resourcing misalignment (not ready when you need them to be, don’t follow your prioritization, don’t have the right skills for the job)
Objectives are not aligned
- Quality vs. cycle time
Prioritization is broken
- Reverts to first come first served
- Reverts to servicing the squeaky wheel to the detriment of others
- Easily gamed by managers
- Lack of insight into importance of project (sequencing, ordering, dependence on other projects – so just biz value alone doesn’t always provide the complete picture of how integral a project is to a company)
- Subjective measures of value
Governance/Process Sclerosis – to manage the demand these shared service groups often put out draconian service catalogs and do things in an overly bureaucratic way.

The Anatomy of Decisions

2009-08-31T16:01:00.001-04:00

Two posts ago, I wrote about making decisions with very little information -- the bane of top executives. This post outlines the various attributes of a decision.

Attributes of Decisions

Similarity to other decisions made
Scope or impact of decision
Availability of needed information - timeliness
Integrity of available information
- accuracy
- completeness
Ability to determine what would count as information relevant to the decision
Identifying first, second, and third order impact of variables
* Cost of getting it wrong
* Benefit of getting it right
The Mechanics of Decision Making - Structuring the decision elements (the bits of information that are relevant to making the decision)
- composite with weighting (all elements are taken into account simultaneously -- with each assigned a weight towards the outcome)
- strictly sequential (stepping through each element one by one in an ordered sequence - this is the way a flowchart works)
- partially sequential (stepping through sets of elements in an ordered sequence)
  - Intra-set weighting (relative weights within a set of elements)
  - Inter-set weighting (relative weights of the different sets of elements)
  - Sequence order (which set to consider first, next, and so on)

Dimensions of Decisions

Time frame
Level of responsibility
Level of control
Sequence
Resource allocation per sequence
Sensitivity (how much to push and for which output)

Quality of Decisions [from The Logic of Failure by Dietrich Dörner]

Frequency with which hypotheses are tested
Number of decisions per goal
Number of "why?" questions asked
Innovation level of the decision (type of decision made - see the book for details)
Stability index (what the decision maker considers -- see book for details)
Level of fixation on a single thing
Degree of self organization
Facing adversity without shirking
Ability to deal with ambiguity

Reasons for Decisions Going Awry [from The Logic of Failure by Dietrich Dörner]

Trying to control the outcome, not the process

Oversteer - looking at points in time when it's the change from time to time that matters
Not understanding exponential growth or deceleration
Not accounting for time lags between cause and effect

Mental habits

Ritualization of behavior
Coming to believe that there's no rationally comprehensible principle at work
Overgeneralization from local experience

Failing to understand complex system behavior

Interdependencies
Complexity
Internal dynamics
Incomplete understanding of the complete system (the whole behaves very differently from the parts)

Three Types of Software Quality

2009-08-30T18:58:00.000-04:00

There are three basic types of software quality.

Functional Quality -- a measure of what the software does vs. what it's supposed to do
Non-Functional Quality -- a measure of how well it does it vs. how well it's supposed to do it
Latent Quality -- a measure of how well it will continue to do it in the future

I say basic because out of these three types emerge other aspects of quality -- Usability is one such derived quality. Think of it as filling the gap between the what and the how well -- it's how the product is put together to meet the user's needs. This is a mix of Functional and Non-Functional quality (and perhaps even a little bit of latent quality).

Managing What You Can't Measure

2009-08-24T18:24:00.003-04:00

I'm tired of hearing "You can't manage what you can't measure." If you're older than 6, you know Santa is a fiction. If you're older than 11, you don't need me to break it to you that of course you can manage what you can't measure. There's big money in that.

Let me give you some examples of managing without measuring.

Managing Without Measuring

+ Tiger Woods manages his swing quite well when it's windy. He might know the wind speed and direction, but he's not managing the way the wind blows. He's managing his body mechanics.

+ Baseball hitters manage to hit the ball without measuring anything. Yes, their eyes and brains are processing information; if you want to call that "measuring", then fine, but realize that just makes the original claim trivial -- "you can't manage without your brain processing information". OK, I won't argue against that -- even someone in a coma fulfills that condition.

+ Writers manage to finish screenplays without measuring anything.

+ Bridge players manage to win hands without measuring anything. (I'm cheating a bit, but read on.)

So there's plenty you can manage without measuring.

What About Managing Well?

Rankled yet? "This is not what managers do!", you say. "Managers manage people, they set targets, they measure progress towards those targets, and they alter course and speed depending on the results of their measurements." Yes, I agree. Strictly speaking, you don't need to measure anything to manage people, but the good ones seem to be measuring things. And by the way, bridge players are measuring something -- the odds that their opponents have particular cards. (I'll come to that in one moment.)

So maybe we should say, "To manage well, you need to measure." Or, "You can't improve what you can't measure." Or, "You can't control what you can't measure."

Now we're getting somewhere. Managing well means improving performance and gaining some degree of control over the outcomes you're interested in.

To improve or control outcomes you have to at least have a measure of the outcomes. Often you need more. If you manage a sales team you might measure completed sales per quarter. If you manage a research team you measure the number of publications per quarter and their quality. Measuring outcomes can range from easy (number of sales in a given period) to hard (the quality of a research paper). But that's a separate issue.

To improve or control something you often have to measure more than just outcome metrics. You have to measure process metrics. If you're looking for completed sales, then you need to research companies, send emails, call companies, and meet with prospects. If you're not hitting your sales numbers then you can ask why not -- am I making enough calls, sending enough emails, meeting with enough prospects?

Process metrics help you figure out where things might be going wrong. If you measure the right things, you might even be able to catch things early, before it becomes a full-blown problem. And process metrics themselves can be broken down into objectives and process metrics, and so on, iteratively. You'd be a fool to run production in a manufacturing plant without a lot of these layers of metrics.

You can be measuring all these things quite nicely and still not be managing well. The reason: you don't know what really influences outcomes and how much.

To have a feel for that is to have a fairly complex causal model -- a set of drivers that interact with each other and the outcome variables to produce the outcomes you're looking to improve or control.

So, "You can't manage what you can't model." But modeling is hard.

* It's hard to know what to measure (there are lots of things you could measure -- how do you pick the right ones?)
* It's hard to know how to measure it (quality of an idea? an author's degree of influence? complexity of an ERP system?)
* It's hard to know if the causal model is valid (do the metrics you've strung together really influence the outcome, and if so, by how much?)

Once you've gotten past these issues, you're ready to manage! Congratulations!

How do the best managers do it? Sure they have models. But the best managers are really good at two things:

* Having a great feel for what influences what and by how much,

and most importantly,

* Having a great feel for what to look for to know that you're not on the right track to the goal (either you're off in direction or speed)

This second attribute -- knowing whether you're off course and what to do to get back -- is what good CEOs get a lot of money to do.

Why? Because their decisions are often made in conditions where:

* There is very little information
* The situation is not very similar to ones they've faced before
* The stakes are high
* There's very little time to make the decision

If you find yourself in this situation time and again, there's only one way to manage. You've got to know quickly that you're off track and know what will get you back on. People who can do this for a living (justifiably) earn gobs of money.

Cloud -- Is That Something You Might Be Interested In?

2009-08-20T10:22:00.017-04:00

This morning on Slashdot, there was mention of an Australian team that studied the performance of the Amazon, Google, and Microsoft Clouds. It reminded me of Bob on Entourage.

The results are not surprising. The on-demand cloud services from these companies "suffer from regular performance and availability issues."

Now, not to make too much of this -- we already know that blazing performance on the cloud is neither a promise these vendors make nor an economic reality. After all, if you want cheap AND scalable, something's got to give.

But you can be prepared.

If you could precisely measure the performance and availability of an application on the cloud, would that be something you might be interested in?

If you could do this before you migrated to the cloud, would this be something you might be interested in?

If you're a vendor of Cloud services, would you be interested in tracking not just usage, but quality of service?

Well, you can.

1. If you manage an IT organization, use CAST before you move one or more of your applications to the Cloud. A CAST analysis will determine which applications are ready for migration to Cloud and vet the performance of those applications before you put it on the cloud. Once on the cloud, CAST enables you to painlessly monitor your application's performance to ensure you are not wasting your money.

a. Understand how well the application will perform, measure robustness, performance, and security. (Cloud hosts can kick you off the cloud if your application puts others on the cloud at risk.)

b. In measuring quality, CAST can also quickly highlight and measure the drivers of application costs.

c. If cloud is a cost cutting measure, use CAST metrics to make sure you're not burning more MIPS, using more memory, and transferring more data than you should.

2. As a SaaS/Cloud vendor, use CAST for competitive differentiation.

a. Measure and communicate the quality of your SaaS/Cloud environment to current and potential customers.

b. Use CAST’s quality metrics to demonstrate the measurable cost of quality of your services.

I'd be glad to tell you more - just email me. Or go here.

Now Bob might be a parody of himself, but he really gets to the core of what matters. In software, it's the only thing that matters in the end is the product, the stuff, aka the code. It's so difficult to measure that people get frustrated.

But it's something you should be interested in.

Got SOA? Get CAST!

2009-08-14T14:08:00.006-04:00

Four Reasons for Using CAST for SOA Development, Rollout, and Governance

1. Stop critical single points of failure. A failure in a single SOA component can have an impact on multiple applications. Hence, quality is critical to SOA components.
a. CAST automatically analyzes applications and components against more than 800 industry best practices to identify quality lapses in your SOA repositories,
b. CAST provides actionable guidance on how to improve the quality of your SOA components.

Question: Do you know how to reliably detect and stop critical single points of failure?

2. Manage application performance across the multiple technologies and tiers spanned by your SOA systems. SOA repositories are typically built on a wide range of technologies. Performance problems are hardest to detect at the interfaces between tiers and technologies. CAST’s 28+ different native analyzers cover most technologies and platforms. CAST analyzes interactions between components across different layers and technologies to ensure each individual component, and the application as a whole, is of high quality.

Question: How does your data and application logic behave when crossing technology tiers?

3. Give your architects and developers practical guidance. Governance is the key to successful SOA implementation. However, governance is often an "ivory-tower" activity with very little practical guidance for day to day operational decisions. Strict conformity to SOA component development guidelines is key to a successful implementation of SOA. CAST automates the enforcement of these guidelines.

Question: How can architectural rules be written to provide practical guidance for developers?

4. Make service reuse a reality. The promise of service reuse, and the quantum leap in productivity it can generate, is quickly ruined by unprincipled service explosion. The result is a spaghetti bowl of services with many different variants of the same service, and developer confusion about what to reuse.
a. CAST automatically generates detailed information on service inputs and outputs, giving developers and architects a practical way to see the degree of similarity between any two services
b. Developers and architects can use this practical information to drive decisions around service modification, consolidation, or elimination.
c. CAST's automated visibility into service attributes makes service reuse a reality.

Question: How do you keep the explosion of services in check?

Why the CAST AIP is Cool -- Five Reasons

2009-08-10T05:01:00.006-04:00

As you know, I work for CAST. I work here because our product is really cool.

Let me tell you why it's cool and I think you'll agree as well.

The CAST Application Intelligence Platform (AIP) is cool because it goes beyond what’s available in 5 groundbreaking ways.

1) The CAST AIP objectively measures software quality, complexity, and size – attributes that have traditionally been very slippery to quantify. (Think about this for a second -- it happens to be true.)

2) CAST’s coverage of languages, platforms, and technologies is unmatched (30+ languages, all major platforms and technologies, and all major packaged software including SAP and Oracle).Trying to simulate what the CAST AIP does with open source applications or other existing products is an exercise in frustration. Not only do you end up with a patchwork quilt of systems, they don't really do what you really want them to do -- reliably detect the problems that lurk in the interfaces between technologies and platforms. Only the CAST AIP can do this.

3) Granular, surgical drill downs. The CAST AIP gives you multiple views into the end-to-end (developer, architect, project manager, business manager) and precise guidance on fixing the root cause of the problem. CAST AIP is a great example of how insight is used to guide effective actions.

4) Quantifying productivity and the effectiveness of process improvement. The very metrics that are used to measure software quality, complexity, and size can be used to track the effectiveness of improvement efforts. You can read more about it in a previous post.

5) CAST AIP goes beyond performance testing by:

Automatically tracing application performance problems to the root cause. Testing teams can find problems more reliably, fix them faster, and fix them once and for all using the precise guidance CAST AIP provides.
Finding performance bottlenecks that go beyond those uncovered in traditional performance testing by revealing problems that lie hidden at the interfaces between language and technology tiers.
Uncovering future problems, not just present ones. Performance testing doesn’t tell you how something will perform in a few months when production conditions have (invariably) changed. It won’t tell you how easy it will be to modify the system to meet a pressing business need. And it won’t tell you how difficult it will be to transfer support to another team (internal or external). CAST AIP can do all of the above.

When something is this different it takes a while to appreciate the power of it.

How to Name Things

2009-08-09T11:35:00.002-04:00

I name things. I do so frequently. I come up with lots of names every day for our cat.

When I'm more reflective about naming, I use this checklist.

The WHAT

What is it?
What does it do?
How does it do it? – The critical active ingredient.
What is *the* thing that makes it unique?
What are the benefits?
What are the benefits of the benefits?

The Mechanics

Is the name easy to say?
Does it need a tag line?

6 to 8 words max
Personable, lively, clever, but not an inside joke

Does it lend itself to a good acronym?
Does it escape the Chevy Nova problem?

The Emotion

Does it make me think of good things?
Does it make me proud to say it?
Does it make it me care enough to act?
Does it tie into my rituals?
Does it inspire me?
Does it motivate me?

P.S. When you take the Dulles Airport access road from Washington DC to (where else) Dulles Airport, you see a lot of dot com (er, Web 2.0) company names plastered on buildings. They're a great source of good as well as bad names.

A good one is "Oracle". They're so huge and ubiquitous that we don't really pause to reflect on it any more, but "Oracle" is a great name for a database company!

On the other hand "Simplexity" (I'm not making this up) is a terrible name! Look for it the next time.

A Recipe for Quantifying the ROI of Process Maturity Improvement

2009-08-07T11:08:00.002-04:00

Types of Process Management Frameworks

What should I do? -- Process Definition Frameworks (Associated Metric Type = Objective or End-Result, or Benefit)
1. ITIL
2. MOF
3. ISO, etc.
How well am I doing it? – Control/Audit/Maturity Frameworks (Associated Metric Type = Operational Level or Quality Indicator)
1. BS 15000
2. CMMI, etc.
How can I improve it? – Process Improvement Frameworks (Associated Metric Type = mix of above two metric types)
1. Six Sigma
2. TQM
3. Lean, etc.

A Recipe for Quantifying the ROI of Process Improvement -- the bang for the buck on improving the repeatability and quality of processes:

1.Define the end-result metric (e.g. support cost per year in $)
2. Define the process performance metric (e.g. defect injection rate)
3. Quantify the end-result metric as a function of the process performance metric -- e.g. support cost per year = f(defect injection rate)
4. Define processes and break down process activities (e.g. testing process)
5. Define activity maturity levels -- what it means to be at maturity level 1, level 2, etc.
6. Quantify the change in performance metrics due to change from one maturity level to another.
7. Use (3) to calculate the benefit of moving from one maturity level to another.
8. Quantify the cost of moving from one maturity level to another (one-time and ongoing cost categories are: organizational change, business process change, software licences, productivity loss, administrative costs)
9. Divide (7) by (8) to calculate the ROI of process maturity improvement

Antonella and Software Requirements

2009-08-03T06:09:00.001-04:00

Antonella is 28. She's attractive, lives in Rome, and loves to go clubbing.

I'd like to know Antonella! But it's not what you think.

Antonella is a fiction. But a useful fiction. The designers at Ford Motor Company use her (or the idea of her) to guide their design decisions and resolve conflicts. As Phil Patton, the author of an article on Ford says:

"Ford is using characters like Antonella to bring a human element to the dry statistical research drawn from polls and interviews. Based on psychological profiles, these characters are a more modern version of the 'theme boards' that designers once covered with snapshots and swatches of material to inspire a design. They are also like avatars, those invented characters used in online games and forums to symbolize a participant’s personality."

Patton quotes Murat Yalman, Ford's director of global advanced product strategy: “You get a common focus for everyone from the clay modeler to the chief executive.”

Can this technique work for eliciting software requirements and keeping them up to date as the design evolves? Do Antonellas exist for software projects?

I'm old fashioned. When I think about getting requirements right, I think about getting to a set of features (not more than 20 to 25). I then draw each feature out into a set of use cases, and turn each use case into a bunch of requirements.

I could definitely use Antonella at each of these stages, especially at the requirements satisfaction stage where all the difficult conflicts arise about how best to fulfill a feature through a set of requirements.

Or at least I thought so, until I read this in Stephen Few's 2006 book Information Dashboard Design. Few writes:

"In a genuine attempt to please their customers, software engineers focus on checking all the items, one by one, off of lists of requested features. This approach makes sense to technology-oriented software engineers, but it results in lumbering beasts. Customers are expert in knowing what they need to accomplish, but not in knowing how software ought to be designed to support their needs. Allowing customers to design software through feature requests is the worst form of disaster by committee." [My emphasis]

The distinction is between knowing what I as a customer need to accomplish, versus knowing how a tool that helps me accomplish that goal is to be designed.

Makes sense. In software design, especially in Agile development methodologies, are we veering too far in the direction of letting customers design the tool itself?

Can Antonella prevent us from going too far? Maybe we should all get to know Antonella.

Feeling Lucky? The Prequel aka the Back-Testing Problem

2009-08-01T16:37:00.052-04:00

A couple of years ago, I took the 14-hour Qantas flight from Los Angeles to Sydney and back.

For a slow reader like me, 14 hours is the perfect time to polish off a book from cover to cover. On the way to Sydney I read Thomas Schelling's Micromotives and Macrobehavior. The hair on the back of my neck stood up; I got goosebumps!

Schelling's analysis is so clean and simple, yet packs a tremendous conceptual wallop. I was instantly reminded of Einstein's Investigations on the Theory of Brownian Movement and Feynman's QED. These books make each step seem so simple, yet when you string them all together, the result is substantial, deep, and beautiful.

Schelling's book took me back to the first time I laid eyes on Euclid's axioms. Before I knew it, we had landed and I was in a long taxi line that was winding it's way through a temporary building outside the terminal.

On my way back to Los Angeles, I read Nassim Taleb's Fooled by Randomness. It's a relatively slim book. But it kept me occupied the whole flight. In fact, it got me angry. I was cursing out loud! (I come from the How Late It Was, How Late school of cursing.)

I think it was Bertrand Russell who wrote this one-line book review: "I should think the covers of this book are too far apart." That summed it up for me. Fooled by Randomness could have been a great book. Instead the tragically hip pseudo intellectual poseur shtick beat the life out of it.

Here's what it could have been.

A Precis of Fooled by Randomness

Any data set can be “explained” by an infinite number of patterns (Choice and Chance, by Brian Skyrms). [This has hugely interesting implications -- more on this at the end]
Use Monte Carlo analysis to uncover the role of randomness:
- Rule out alternative explanations (ones that have very low probability)
- Determine the relative strength of the current explanation
- Find interdependencies of variables in certain permutations
- Find missing variables
Interesting results from probability theory that should inform your reasoning:
1. Dependence of probability on time scale
2. Counterintuitive results due to skewness and asymmetry
Common mistakes in probabilistic inference (the best summary of results is Scott Plous' The Psychology of Judgement and Decision Making)

Main Point: Even the most savvy Wall Street traders make huge mistakes when they reason about statistics. In particular, they grossly underestimate the role of sheer luck in the results they achieve.

Now that would have been a great book! Not wholly original, but highly informative.

To give you a sense of what this randomness is like (and how Monte Carlo analysis can illuminate it), here is a quote from LeonardMlodinow ( The Triumph of the Random by , WSJ July 3 - 5, 2009 Page W1)

"A few years ago Bill Miller of the Legg Mason Value Trust Fund was the most celebrated fund manager on Wall Street because his fund outperformed the broad market for 15 years straight. It was a feat compared regularly to DiMaggio’s, but if all the comparable fund managers over the past 40 years had been doing nothing but flipping coins, the chances are 75% that one of them would have matched or exceeded Mr. Miller’s streak. If Mr. Miller was really merely the goddess of Fortune’s lucky beneficiary, then one would expect that once the streak ended there would be no carryover of his apparent golden touch. In that expectation Mr. Miller did not disappoint: in recent years his fund has significantly lagged the market as he bet on duds like AIG, Bear Stearns, Merrill Lynch & Co. and Freddie Mac."

So how lucky are you? To find out, read the sequel.

Post Script (Added July 30, 2009)

An excellent example of how a data set can be "explained" by any old hypothesis appeared in the Wall Street Journal earlier this month (Data Mining Isn't a Good Bet for Stock-Market Predictions by Jason Zweig).

Here are some excerpts from the article that make the point (my emphasis) -- enjoy!

An entertaining new book, "Nerds on Wall Street," by the veteran quantitative money manager David Leinweber, dissects the shoddy thinking that underlies most of these techniques.

The stock market generates such vast quantities of information that, if you plow through enough of it for long enough, you can always find some relationship that appears to generate spectacular returns -- by coincidence alone. This sham is known as "data mining."

Every year, billions of dollars pour into data-mined investing strategies. No one knows if these techniques will work in the real world. Their results are hypothetical -- based on "back-testing," or a simulation of what would have happened if the manager had actually used these techniques in the past, typically without incurring any fees, trading costs or taxes....

Mr. Leinweber got so frustrated by "irresponsible" data mining that he decided to satirize it. After casting about to find a statistic so absurd that no sensible person could possibly believe it could forecast U.S. stock prices, Mr. Leinweber settled on annual butter production in Bangladesh. Over an 13-year period, he found, this statistic "explained" 75% of the variation in the annual returns of the Standard & Poor's 500-stock index.

By tossing in U.S. cheese production and the total population of sheep in both Bangladesh and the U.S., Mr. Leinweber was able to "predict" past U.S. stock returns with 99% accuracy.

The Problem of Induction is the problem of demonstrating that the future will be like the past. It can't be done via deductive argument (because by definition, induction takes a leap beyond what is contained in the premises), and it can't be done via induction because that would be circular.

But even if you could solve the problem of induction and demonstrate that the future will resemble the past, it won't help solve the back-testing problem. To solve that, you have to be able to show in what ways the future will resemble the past. There is a super-interesting problem lurking here that goes by the name of the Grue Paradox.

To summarize: back-testing alone is insufficient to demonstrate the validity of a hypothesis. It's insufficient because I can cook up an infinite number of hypothesis that will fit the pattern in question. I need a way sort out good hypotheses from bad.Without that "goodness filter" I don't know if the "pattern" identified will repeat in the future.

Why Developer Tools Stick

2009-07-30T10:51:00.006-04:00

Some tools are very popular with software developers while some just fizzle out. What accounts for this difference?

There are lots of variables that determine the stickiness of a tool (or even methodology). Here's a sociological explanation for what sticks and why.

For something to become sticky as a development tool or methodology, it must have two kinds of utility -- operational, political, and personal. (That's my hypothesis.) Here's how each breaks out.

Operational Utility - making it easier to do the job
1. Ability to improve code quality
2. Ability to improve development productivity
3. Ability to improve communication and coordination
Political Utility - making it easier to defend how software gets made and delivered
1. Symbolic – signal it sends to other developers and management
2. Support/Confidence it provides that we’re doing things in a professional way
3. Defense/Rationale – ability to use it to present, elaborate, and justify the work done to managers and senior executives (metrics, artifacts that show progress and establish that a robust methodology is being used)
Personal Utility -- ability to enhance resume.

Have I missed any? Which ones are necessary? Which ones are sufficient?

Don't Give Up on Software Metrics

2009-07-22T22:11:00.013-04:00

Is software management obsolete? Yes, according to Tim Berry, President and Founder of Palo Alto Software.

You can read my comment on his blog. Here's a fuller version.

I read Tim's post along with Jeff Atwood's post on Coding Horror and Tom DeMarco's article. There are a number of cool ideas in this set of articles, and it's helpful to separate them.

So here goes.

1) Building software is a creative activity.

2) Most projects, despite the controls, metrics, and process, end up being completed by the sheer passion, heroics, and expertise of a few. (I'm reading a bit more between the lines on this one, but it's a charitable interpretation.)

3) In a typical software project "nobody can see the whole elephant."

4) A software metrics paradox: only projects that don't deserve or need the scrutiny end up getting them.

5) To figure out how to build software without a lot of metrics fanfare, just pick the projects with very high ROI -- the return on these will dwarf the investment, so you don't really need to pay very close attention to the investment (i.e. the cost to build).

6) It's about time we abandoned this charade and stopped using software metrics (on most or all of our projects).

1, 2, and 3 are factual claims, 4 is descriptive (e.g. like Moore's Law), and 5 and 6 are prescriptive.

I agree with 1, 2, and 3. As Fred Brooks puts it in his book, "The Mythical Man Month", "The complexity of software is an essential property, not an accidental one". And he goes on to explain how this complexity manifests, and why it is essentially different from, say, building a skyscraper.

But despite that, I think it's possible to impose the right controls throughout the software development life cycle. Let me expand on this just a bit.

Throughout the course of development, a large number of developers, architects, IT workers, business users, and managers make hundreds of independent decisions which need to come together in a coherent fashion for the end product to work well.

So, “software design is not an orderly top-down process. It is a collection of multiple decisions made at varying levels of abstraction, brought together to satisfy an overall business goal.” (This is Dr. Bill Curtis, co-author of the CMM framework.)

As the code base grows linearly, architectural complexity grows exponentially! Performance bottlenecks multiply but become very hard to detect. There is no human or team of humans capable of having a comprehensive end-to-end view of how the application is put together to satisfy its long list of continually-evolving requirements.

What's a project manager to do? You can't blame them for doubling down on process! But that's the wrong type of metric. What you need in this case, in addition to process metrics (time spent, adherence to coding standards, etc.) are PRODUCT metrics. In particular, measures of the QUALITY of the evolving product.

Think about this for a second. Software is the only product where quality is equated with meeting functional specifications.

You don't make this mistake with a Toyota. Its quality comes not just from having four wheels and a transmission. Chevy Cobalts have those too. The quality of a Toyota comes from something more -- it's HOW WELL the four wheels and the transmission work when you drive the car. How well do they hang together when you hit nasty potholes? How do the parts wear with time?

How well something works is closely tied to the process by which that something is made. There's usually a process for putting things together, and software engineering has grasped at process as a panacea.

Here's the mantra: I have a repeatable process in place, I will end up with a high-quality software product, I have a repeatable process in place...

Unfortunately, it doesn't work this way. Think of all of the FDA approved products that have killed people, the planes that have had all their maintenance checks but crashed, and the Airbus A380 that couldn't be put together because the parts made by different suppliers wouldn't fit together.

Having a repeatable process in place isn't by itself enough. In real life, most processes outside of manufacturing lines are difficult to make repeatable.

Repeatable means same inputs result in same outputs. But in a software engineering process, cost and time pressure, variance in expertise levels, the (un)availability of expertise, the (un)availability of input information -- these can and will lead to a wide variance in the output of a process.

Making software is like making the Airbus A380 -- hundreds of people making independent decisions that have to come together in the end as the finished product. Indeed, it's even harder when it comes to software because the product is not a tangible thing like an airplane part. And the kicker: when it all comes together, there's no foolproof way to know how this thing is going to work in the real world!

I work for a software company called CAST -- we define and precisely measure the quality, size, and complexity of large-scale software systems as they evolve. No human can see the whole thing end to end, but CAST can. Write me if you like, I can show you exactly how we do it. Bottom line: There is a way you CAN see the whole elephant.

The lesson is PRODUCT metrics, not PROCESS metrics. And humans can't do this -- but an automated system can.

Point 4 may be right, but unfortunately, it doesn't lead to 5. We don't get to choose the projects we work on, and DeMarco's guidance is simply not practical.

Finally 1+2+3 is not reason enough to believe in 6. We can get complete visibility over the entire software system, and we can measure it the way engineers measure their output.

Give up on process metrics if you like (and if you can in your company), but don't give up on controlling software projects - use product metrics to take control!

Tech Spending -- Where is it Going?

2009-07-20T10:03:00.000-04:00

Phyllis Korrki reports in The New York Times (Sunday, July 19), that "a little over a third of businesses and agencies plan to increase their technology spending in the next six months, with more medium and large-size companies anticipating gains than small ones."

The article goes on to say, "More than half the technology managers for the federal government expect increases in their spending budgets."

A couple of points of note in the data.

First, for the 7% who respond that IT investment hurts their financial performance, I wonder why?

Is it just that the choice of investment is poor (misjudged the ROI of the investment), is the execution poor, is the end result poor, or all of the above?

If the end result is poor, it could be due to one or these factors:

* The automation doesn't work as well as it should and hence scares away customers?

* The automation works well, but it just turns customers off?

* The automation makes internal staff less productive? (e.g. because they now have to do some extra things)

Second, if you are spending more, where is that spending going?

Effort is Precious

2009-07-15T11:20:00.004-04:00

There's a big bias towards skill in our culture -- smart and creative trumps hard work and grinding it out day after day.

We come to rely on our skills and often think too much of them. Studies have shown that most successful people over-rate the part that skill and creativity play in their success.

As for me, it's been all about being at the right place at the right time. Sheer luck and accident.

But what about effort? What about the joy of working hard, grinding it out? It can be very valuable. Especially when no one can -- or is willing -- to match that effort.

A few months ago, Malcolm Gladwell wrote about a basketball team with no skill to speak of. They won big with sheer effort alone. The kind of effort that drives skilled opponents mad. The kind of effort that no sane person is willing put himself through.

Remind you of outsourcing? The hundreds of thousands in India who can and are willing to grind it out while you spend your time reading the New Yorker? Well, they're just a commodity -- but a very precious one!

In his article "How David Beats Goliath" (New Yorker, May 8, 2009), Gladwell writes:

"We tell ourselves that skill is the precious resource and effort is the commodity. It's the other way around. Effort can trump ability because relentless effort is in fact something rarer than the ability to engage in some finely tuned act of motor coordination."

So what makes a commodity a commodity? What are the attributes that separate a commodity from a non-commodity?

1. Ease of Access (easy to procure vs. very hard to procure)
2. Concentration of Supply (single source vs. large number of sources)
3. Variability
1. 3.1 Outcome Variability: If the item is a process, then repeatability of the process (high variance vs. zero variance). Commodities have low outcome variability.
2. 3.2 Compositional Variability: If the item is a thing, then the level of purity (pure vs. lots of processing needed to make it pure). Commodities have low compositional variability.
3. 3.3 Pricing Variability: How readily can it be componentized (and hence, priced)? (easy to componentize vs. impossible to componentize). Commodities have low pricing variability (not necessarily low price, but low pricing variability)
4. Marginal Cost of Production (connection of cost to amount): cost of making one more is virtually zero. Small change in production costs for large changes in output. Commodities have low marginal production cost.
5. Marginal Cost of Labor: labor can be added at virtually zero cost. (E.g. a delivery person gets paid the same amount even when time taken to deliver varies.)

Some Observations

Of these attributes, the central one is 3.3 -- how well can it be componentized. Most often, 3.3, is closely related to 3.2 and 3.1. It is likely that if 3.3, then (3.1 or 3.2), i.e. the latter are necessary for 3.1.

4 and 5 could serve as core criteria as well.

1 and 2 are red herrings -- if you plot a commodity along these dimensions, you'll get a scatter plot; so 1 and 2 don't have any segmentation power.

There is a difference between having a well defined surface structure and being a commodity. Commodities will have a well defined surface structure AND a well defined deep structure. It's the absence of a well-defined deep structure that makes the work of advertising agencies, lawyers, management consultants, and industry analysts non-commodities. For example, take company/industry analyst reports. These are componentizable (it's a specific form factor with particular sections, etc.) and hence have a surface structure, but there's a huge amount of outcome and compositional variability which indicates a lack of deep structure.

The world is such that most segmentation problems have to be tackled this way -- create the net of distinct attributes and then see how much of the right stuff it will catch and how much of the wrong stuff it will let through.

Related issue: segmenting and aligning demand with supply.

If this analysis is right, then a few things follow:

a) a commodity can be expensive

b) a commodity can be rare

c) a commodity need not be easy to procure

d) 5 might indicate that when laborers are readily available and (perhaps, consequently) have no pricing power commodity, the work they do becomes a commodity. There might be a more widespread confusion of commodities with a lack of pricing power. A branding company works to a well-defined goal but has lots of pricing power; a laborer in a South African diamond mine does the same but has no pricing power.

e) commodities are not to be confused with well-defined objectives or end products that result from combining labor and capital. That's why the crux of the definition is 3.3. To call any individual item or thing (or SKU) a commodity is to destroy the meaning of the term.

The View from the Other Side

2009-07-14T20:23:00.000-04:00

Most people hate marketing emails -- I love them!

Galileo said that the world is an open book written in the language of Geometry. The internet is an open book written in the language of marketing spam. There's no better way to stay on the pulse of this wild, weird thing we've come to love - the internet (or email, to be a bit more specific).

So why this paen to spam? Well, a few days ago, I got an email from WebEx, you know the web meeting software company now part of Cisco Systems.

It's really quite good -- and having sent many of these sorts of emails myself, I put myself in the shoes of a straightforward, but understanding recipient. Here's the email and my response.

Subject: WebEx / A Few Minutes To Catch Up

Jitendra,

It's been a while since we've been in touch. Since we last spoke, WebEx has introduced new applications, services, and flexible pricing models that have been designed for businesses like yours. I would like to share some creative ways that companies like yours are using WebEx services to cut costs, increase revenue, increase lead count, and make their teams and departments more efficient.

Can we connect later this week or early next?

Please call or reply to this email with a couple of dates and times you are available in the next week.

And my response:

Hey – I know this might be an automatically generated email from your campaign system, but I wanted to let you know that we’re already using WebEx at CAST. It’s likely that your CRM system is not up to date on this.

If it’s upsell you’re looking for, then I’m afraid I don’t make any of those purchasing decisions. :-(

As a marketer, I enjoy getting these emails (really!). The message is appealing. I have two thoughts that you guys might want to consider on future campaigns:

1) Leaving out the graphics logo – it prompts a security warning that might reduce responses.

2) On the message itself: I get it that I can cut costs with WebEx, but a phrase on how I can increase revenue would be great. It might be “increase lead count”, but I bet most people don’t know what that means. Also, IMHO, most people don’t seem to care that much about making their internal team more efficient *unless* it means cutting headcount.

Anyway, sorry for not doing my part on the revenue front.

Regards,

Jitendra

Be kind out there!

Why PPM Tools Mislead

2009-07-12T14:40:00.002-04:00

There are a lot of Project Portfolio Management (PPM) tools out there. And there's a lot of hot air about the distinctions between PPM, ALM, PLM, and who-knows-what-other-M.

Bottom line: they don't work. Certainly not in the way they are advertised to work. Not even close.

Over the last 7 years I've heard large companies complain about three main things.

Sticker shock -- the up-front cost is too high.
Data integration is difficult -- doesn't play nice with the systems I have (e.g. defect tracking, resource management).
The customization conundrum -- I need to change it to fit my processes but somehow that starts a vicious circle of change that never stabilizes.

I'll focus on a narrower problem. It's very hard to get the quality of data in these systems up to snuff. And even if you do (a big if), the outputs can still mislead.

To the extent that your decisions depend on the accuracy of this data, you're sunk.

The reasons for this are not hugely novel - they are a priori points that apply in large part to all quantitative models (I'm assuming PPM tools are a subset of the class of quantitative models -- nothing very controversial.)

So here we go -- there are some repetitions in the list and that's purely on purpose.

If you'd rather read a graphical version of this, check it out as a mind map on MindMeister!

Root Causes of Misleading Results Generated By PPM Tools

1. Low Data Quality

1.1 lack of completeness
1.1.1 lack of motivation
1.1.2 lack of time
1.1.3 data not available when needed
1.1.3.1 lack of interfaces/connections
1.1.3.2 tool not optimized for the environment
1.1.4 data too difficult to get
1.1.5 gaming the system
1.1.6 lack of expertise
1.1.7 no place to enter relevant data (e.g. PTO hours)

1.2 lack of timeliness
1.2.1 lack of motivation
1.2.2 lack of time
1.2.3 data not available when needed
1.2.3.1 lack of interfaces/connections
1.2.3.2 tool not optimized for the environment
1.2.4 data too difficult to get
1.2.5 gaming the system
1.1.2.6 lack of expertise

1.3. inaccurately entered data
1.3.1 lack of motivation
1.3.2 lack of time
1.3.3 data not available when needed
1.3.3.1 lack of interfaces/connections
1.3.3.2 tool not optimized for the environment
1.3.4 data too difficult to get
1.3.5 gaming the system
1.3.6 lack of expertise
1.3.7 sensitivity to sequence in which data is entered (e.g. one input has to be set before the other but isn't)
1.3.8 sensitivity to interdependencies (e.g. two inputs need to be entered in concert but are typically set independently)
1.3.9 Misaligned map of resources to tasks (things don't quite fit neatly into pigeon holes when mapping resources to tasks). This results in the wrong amounts ascribed to the wrong things.

1.4. imprecisely entered data
1.4.1 lack of motivation
1.4.2 lack of time
1.4.3 data not available when needed
1.4.3.1 lack of interfaces/connections
1.4.3.2 tool not optimized for the environment
1.4.4 data too difficult to get
1.4.5 gaming the system
1.4.6 lack of expertise
1.4.7 sensitivity to sequence in which data is entered (e.g. one input has to be set before the other but isn't)
1.4.8 sensitivity to interdependencies (e.g. two inputs need to be entered in concert but are typically set independently)
1.4.9 Misaligned map of resources to tasks (things don't quite fit neatly into pigeon holes when mapping resources to tasks). This results in the wrong amounts ascribed to the wrong things.

2. Misleading Outputs (will occur even if data quality is perfect)

2.1 Input interdependencies that are hidden -- for example, the tool calculates an output -- e.g. the existence of a particular bottleneck -- based on three inputs. Depending on how sensitive the output is to the existence of and the accuracy of each input variable the calculated output can turn out to be useless. This is a pernicious because it is hidden to the user of the tooland puts the user in the dangerous position of making bad decisions due to algorithmic subtleties. In other words, the user has no grip on the amount of variance in the result. This makes it dangerous to base decisions on this output.

2.2 Divergence of actual resource management processes from the processes dictated by the tool

2.3 When multiple models are used (most PPM tools contain project, portfolio, resource management modules), the inputs and outputs of each of these sub-models can have unknown interdependencies that drive systematic or unsystematic error.

2.4 Parameters or constants of the model change and these are not part of the experiential feedback loop for the model (this could include weights of variables or strengths of connections between variables.
2.4.1 internal factors (new project types, new resource types, new business models, new resourcing strategies, ...)
2.4.2 external factors (macroecononic, microeconomic, exogenous - e.g. Gladwell: "your algorithm doesn't accommodate the fact that the Russian government decides to default on its loans."

2.5 Variables of the model change or new ones need to be added
2.5.1 internal factors (new project types, new resource types, new business models, new resourcing strategies, ...)
2.5.2 external factors (macroecononic, microeconomic, exogenous)

2.6 Algorithm changes
2.6.1 internal factors (new project types, new resource types, new business models, new resourcing strategies, ...)
2.6.2 external factors (macroecononic, microeconomic, exogenous)

2.7 Model changes (i.e. both variables and algorithm change)
2.7.1 internal factors (new project types, new resource types, new business models, new resourcing strategies, ...)
2.7.2 external factors (macroecononic, microeconomic, exogenous)

And lastly, what a PPM tool (or risk model) cannot by itself do:

* get you executive support
* figure out the process you should put in place
* ensure that the skill needed is actually the skill listed in the input
* ensure data quality
* make decisions on what actions to take

Whew! Thanks for getting through it! Check out the MindMeister version which might be easier to read.

CFOs are Easy

2009-07-11T09:56:00.000-04:00

To sell advice to, that is…I wouldn’t know about the other thing.

A few weeks ago, I was standing in an old friend’s flat in London, looking out onto the street, sipping a delicious drink before dinner. Lovely flat, beautiful street.

This friend sells research products to CFOs of the largest companies in the world. He used to sell similar products to CIOs. We got talking about what the differences are between these two groups and why selling to one might be easier or harder than selling to the other.

Why it’s easier to sell research to CFOs than to IT Executives

* Urgency for a CFO is event driven and the events are clear for all to see — they’re news events about regulation, economic conditions, etc. Urgency for IT execs is event driven, but these are hidden to company outsiders and tied to specific IT initiatives.

* The CFO vendor landscape is less competitive than the IT vendor landscape (probably more so when it comes to research on what other companies are doing), not just economic and financial indicators.

* The gap from information to action/utility is smaller for CFOs than IT executives. It might be an issue of how much it takes to actually be able to do something with the information.

This gap between information and action is worth thinking about some more.

A few months ago, someone told me of a three-way distinction that Matt Olson, a research luminary at the Corporate Executive Board uses. It’s a way to qualitatively assess the gap between a piece of advice and the action that it requires to turn it into practice and reap the intended improvement.

Matt Olson’s Taxonomy of Advice

* Just add water — Very little work needed to turn advice into practice and reap the intended improvement — it’s plug and play.

* Just add money — Same as above, but needs to be greased with sufficient funds to get it rolling.

* Just add blood — This is the tough one. We’re talking labor, change management, project management, delivery schedules, and all the attendant headaches. Sweat. Tears.

My suspicion is that most IT research/advice falls into the “just add blood” category — as a CIO, I can’t just run with it. And that makes it worth very little.

Do You Feel Lucky? Well, Do Ya?

2009-07-09T21:52:00.003-04:00

Clint Eastwood said it and Pete Peterson’s life story is apparently filled with it, but can luck be quantified?

Yes, according to two Business professors, Robert Connolly of the University of North Carolina and Richard Rendleman of Dartmouth. Their work is the subject of a short article in a recent Wall Street Journal article.

By using cubic spline functions on data from every PGA tournament from 1998 to 2001, they’ve been able to separate out the roles played by skill and by luck in golf.

Based on their technique, they can tell you the score that any golfer’s intrinsic skill alone will produce. Scoring below that is due to good luck; above that, and you’re having bad luck. The amount of luck is measured directly in the number of strokes over or under your skill-based score.

That’s fascinating! It got me thinking about other data sets that this type of analysis might be applied to. We all “play” in arenas where we don’t control everything — and often we don’t even know the relevant myriad variables. This technique might be a way to separate out what we do control from those exogenous factors that influence outcomes.

Some areas where this technique might come in handy:

* Strategic business decisions made with very little information
* Stock picking or portfolio balancing
* Predicting the number of gold medals a country ends up with in the next Olympics
* Calculating the optimal capacity of a company’s IT network
* The number of business disruptions a mission-critical IT system causes

and many many more. In general, these are all cases of complex systems with a multitude of interconnected variables of which any player can only influence a subset.

Once we separate out the skill (or controllable) part from the luck part, what do we do? Well, at least now we can make a more informed decision about how much slack to build in to the business case or contract, or how much insurance to buy.

Luck may favor the well prepared; but even when it doesn’t, you’ll now be prepared.

Cloud Computing: Why Now?

2009-07-09T12:48:00.012-04:00

The financial markets have their secular cycles. Technology has its S-curves of adoption. Problem is, it's impossible to tell when that S goes from under-performing little bottom coil to the roaring top of the lucrative cobra head. If you're baffled, have a look at these S curves and you'll know what I'm talking about.

Now, I'm making some large leaps and am most likely off on my history, but the distance from the Object-Oriented buzz of the late 80s to the SOA crescendo of 2004-2005 was a long time coming. The distance between the ASP model in the early 90s to the Utility Computing craze after the dot com bust is finally now turning into the full-throated scream of cloud computing. Seems to be everywhere now. Everyone's on some cloud or the other.

So much so that some major IT vendor is going to make a commercial using the old Rolling Stones anthem as the soundtrack.

So what's changed? Why now? Is cloud just the same old same old? There are a few critical things that have changed.

Reasons for the rise in cloud computing (IMHO)

1. technology advances have driven down the unit costs of:

1. set up

2. scaling

3. maintenance

2. this makes the cloud computing business model viable (vendors can make money even if they are not at full capacity -- similar to less than full load trucking).

3. Browser technology is much much better (with AJAX and other related scripting, tools, and technologies)

4. Web and non-web programming languages and interfaces have improved -- so you can run the front end on a scripted language like python with a back end CICS or mainframe (for example).

The CAST Application Intelligence Platform excels in multi-language, multi-platform environments. Having an end-to-end view of the whole system is invaluable, and CAST can provide that visibility. With the trends of virtualization, cloud, SaaS, PaaS, and related movements, the line between Infrastructure and Applications as functions is blurring. I think this makes CAST's visibility even more valuable.

My friend and former colleague Mark Tonsetic writes about these developments in his blog. It's worth tracking this moving IT target.

Why Performance Testing is Hard

2009-07-08T23:54:00.000-04:00

Why is it hard to close the gap between the testing and production environments? Why is it hard to increase your confidence level that once something is tested it will work fine in production?

Here's why.

1. Some problems only emerge at a certain scale or certain permutation of type and sequence of end user inputs -- even if you could, you don't know this test condition so you don't know how to set it up as a test condition. Because there is no algorithm for this, testing is always hypothesis driven, and there is no mechanical way to systematically reduce the test space. The problem is even harder because an application typical functions with a whole host of other applications in the production environment. Even if there are no functionality dependencies, there can be application server, database server, and middleware configuration dependencies. Bottom line: The right permutations of loading, configuration settings, and user input patterns is impossible to systematically isolate.

2. Information does help -- the problem is (a) there's too much of it and it's not clear what's relevant. (b) Even when you know what's relevant, it's hard to get that information when you need it and often you can only have it after it's too late.

3. Even when it comes to testing by applying loads (otherwise known as "database loading"), life is not easy: it's too hard to get vast amounts of data either due to privacy reasons, or not being able to generate the right kind of data - either it's too random or too uniform and doesn't correctly mimic the real world.

4. Infrastructure Blind Spots for Applications Folks - these are things that have a significant impact on how an application performs in the production environment and can only be fixed by changing the application; yet Applications folks are almost always wholly unaware of the need for such a change.

Type of protocol used (especially in web applications)
TCP window between client and server
Number of application turns during a transaction (application "chattiness")
Data payload transferred during a transaction
Firewalls that the transaction has to travel through

5. In general, there is a coordination problem between Infrastructure and Applications that is due to three fundamental tensions:

Misaligned Incentives: Apps. is rewarded for cutting edge functionality; Infrastructure is rewarded for rock-solid stability.
Misaligned Metrics: Infra has availability and network latency and Apps has metrics of how their apps have passed all the performance tests with flying colors. The problem is that this can all be true, yet performance from a business-user's standpoint can be severely impaired.
Misaligned Resourcing Priorities: When Apps. thinks it's time for Infra to work on their project, Infra has other priorities and vice versa.

Reasons for Poor Software Quality

2009-07-08T22:49:00.001-04:00

Earlier today a friend pointed me to Susann Ulrich's article in IBM DeveloperWorks. It's about the top 10 threats to good software quality.

My colleague, and CAST Chief Scientist, Bill Curtis recently wrote about about the top 5 causes for poor software quality.

They both cite very good reasons, with some overlap.

The root cause of poor quality software is the way in which it necessarily has to be built -- lots of individual developers making hundreds of independent decisions that all have to somehow hang together as a whole. We design top down and hope the design will stick -- much like we do a business case at the start and hope to never revisit it again.

Will more domain knowledge help? Sure. Will better communication between team members help? Definitely. Will more automated testing help? Absolutely. Anything that helps you get your arms around the entire thing so you see it end to end all at once and can watch it evolve - that's what you need to make a substantial improvement on software quality.

Building software is an inherently complicated endeavor -- building it well requires simultaneous and ongoing top-down and bottom-up visibility of the whole thing.