Better BI

Chris Gerrard – Exploring the world of Better Business Intelligence

Commenting on Agile Enterprise Data Modeling

leave a comment »

I’ve been practicing agile BI for over twenty years, and am very happy to see this article by Information Management Magazine’s Steve Hoberman: “Is Agile Enterprise Data Modeling an Oxymoron?

The gist of the article is that agile practices have great value in enterprise data modeling, a sentiment I heartily agree with. This posting recaps my comment on Hoberman’s article.

Agile BI, which includes agile data modeling, deliberately set outs to deliver business value early, often and continuously in the form of meaningful, timely, high quality analytics. Delivering the information business decision makers require when they need it is the whole point of BI. Nothing else matters. Not building elegant data models. Not buying, installing and configuring hugely expensive enterprise BI software. Not building systems that can answer any question that might ever be asked.

Far too many BI initiatives fall into the trap of trying to deliver the universal analytical appliance as their first achievement. As the maxim puts it: “You can’t start with everything; if you try you’ll never deliver anything.”

While I agree with the major thrust of Hoberman’s article there are a couple of misinterpretations of agility that it perpetuates.

First up: “A majority of projects driven by agility, however, lack a big-picture focus and often strive to deliver small slices of functionality within tight time frames, at times redoing and revamping prior work.” While it seems reasonable, even wise, it’s naive and there are a couple of corrections required to this statement.

Redoing and revamping prior work is an essential ingredient element in any agile undertaking. In “normal” agile software development this is called refactoring and is a good thing. Agile BI is no different. Refactoring is always part of the process – agile projects surface and embrace it, make it part of the normal course of events; big-BI project hide it under the covers where it pollutes everything, adds tremendous friction, and invisibly hinders progress.

There’s a hint in the statement that agility projects bias is towards a “lack a big-picture focus”. (yes I know it’s qualified but the smell of allegation lingers). This is a misrepresentation of the true nature of agility. Agility requires a professional awareness of the large scale structures of the diverse elements in the mix. A project that doesn’t do this may wear the clothes of agility, but doesn’t possess its soul. Just throwing functional bits out into the world without regard for the rich complexity of the environment will inevitably lead to a terrible mess. An organization that creates such a mess is unlikely to get it cleaned up, and some poor schlub is usually left trying to continue to work with it while getting blamed for the inability to produce results.

There’s also an argument to be made against using a pre-existing framework to guide agile BI. While it’s incumbent upon BI professionals to recognize the conceptual architectures-business and technical-forming the environment, the paradigm that a pre-formed architecture is necessarily a valuable asset fails to recognize the costs, burdens, and downstream ill effects that result from the principle of simply trying to “fill in the details.” Experience shows that the best architectures are emergent, resulting from the continual refactoring of the developing systems while adhering to suitable and appropriate architectural principles. In this approach, there’s always an intentional architecture that fully supports the real needs of the system, is flexible and adaptable (agility at work!), and can be grown and refined as necessary while never imposing costs not directly related to delivering specific business value.


Written by Chris Gerrard

February 5, 2010 at 11:17 pm

Posted in Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: