We have some really excellent best practices in data warehousing, and we carefully study to be sure we find just the right sources. Be very careful with data transformations to make sure that we are not losing any value from the data as we transform it. We make models that are really going to be useful. With those models we really want to have hefty metadata management.
Nowadays there is master data management and other things that help us to document and beef up the semantics around data. I could go on and on. You have heard this before. This is sacrosanct stuff. And you start contradicting these careful steps, there are people who push back and say no, no, no wait, I have built my career on this rather slow moving but high quality process. So you ask, 'how do we speed it up the data delivery process without losing the quality of data, without losing the intelligence of models and so on and so forth' That's one of the really hard parts.
In so much of the work we do in BI or data warehousing, the data modeling has been front-loaded. We have got this planning process, the requirements gathering, and across the board I am seeing a lot of people just pushing real fast to get to some kind of prototype. And once you get to the prototype that's when the iterations kick in. That's when things we associate with agile kick in.
| #1 Ranking: Read how InetSoft was rated #1 for user adoption in G2's user survey-based index |
|
Read More |
Some people warn, there is nothing more permanent than a temporary solution. So you have to be careful with prototypes because sometimes people, especially business folks, they go well, it’s close enough, deploy it. So we have to be willing to create a prototype quickly but also understand that a lot of them should die away.
You don’t want to put too much of this front-loaded work into a prototype that’s going to die, you wait until the prototypes have survived, and then that’s when you go back and do all that careful data preparation I talked about and all the really careful accoutrements of fine report design, right.
It's hard for me to have a conversation about BI without bringing up big data analytics recently. But the kind of open-ended discovery analytics we are finding with big data recently is agile inherently. And a lot of it does what I was just talking about. You typically grab a lot of raw source data. You typically have one analyst or a small team of business analysts.
We put together analytic data sets very quickly with little or no concern about preparing the data, and really they are trying to build an early data set that will help given the “Aha” moment or answer some business question. And once they do this, it really looks like poor practices compared to what we usually do in data management.
What should survive from that process, they start doing all the careful stuff to institutionalize it because even if something that’s just freeform is this big data analytics. Even that eventually should have a product of BI that does have all the careful semantics and a really good body of reports to go with it.
It just goes to show us that not only is the business environment dynamic but the technology environment s dynamic too. And it's all the more reason for this sort of collaboration and engagement. The word is not just agile. Maybe it’s just about being faster, and that’s all about engagement. It's about people skills.