BI systems need DATA and every business has terabytes of real data which can provide them with the information and knowledge they need to make the right decisions on time. But the key is to turn that data into information in a timely, efficient and effective manner once the WHAT AND WHY questions are answered i.e., what information is needed, what matters and why that is required.  In today’s market, every business is in a RACE. The race to conquer others. The race to generate more gains/profits. The race to foresee the risks early on so that they can be avoided.  So time is of the essence here.

An optimized BI system integrates large volume of external and internal near real time data to allow management to create opportunities by making intelligent decisions after performing predictive analysis of their approach on the business. A good BI System is like a GPS. An effective GPS is one that not only shows you a route to your destination but also guides you when you hit roadblock, gives up-to-date external conditions (constructions / traffic) information, provides multiple routes to choose from, suggests you with alternatives for shorter and fastest routes, predict the total time based on your driving behavior, tells you what to expect next etc. Just knowing the path to your destination is not sufficient. You need to know many other factors during the whole ride to reach destination on time and without any hurdles.

For a good integrated BI system, a good Data warehouse architecture needs to be in place.  Data warehouse architecture is “an integrated set of products that enable the extraction and transformation of operational data to be loaded into a database for end-user analysis and reporting”. Below are the pictorial representations of different “flavors” of DW architectures.

Methodologies used by different architecture:

Kimball’s DW Architecture – Is based on ‘Bottom-UP’ methodology.

Inmon’s DW Architecture – Is based on ‘Top-Down’ methodology.

Dan Lindstedt’s Data Vault DW Architecture – Is based on ‘HYBRID DESIGN’

The first two design methods have some limitations for Data Warehouse layer such as inflexibility and unresponsiveness to the changing departmental needs during the implementation phase, insufficient auditability of data back to its source system, inability to integrate unstructured data, inability to rapidly respond to changes (organizational changes, new ERP implementations) or difficult to load type 2 dimensions in real time. This is where DATA VAULT came in to rescue. Data Vault follows a ‘HYBRID DESIGN’ methodology which follows ‘TOP-DOWN ARCHITECTURE WITH A BOTTOM-UP DESIGN’.

The model is a mix of normalized modeling components with type 2 dimensional properties. In this model, the DW serves as a backend system that houses historical data which is integrated by the business keys. All data ‘good, bad, incomplete’ gets loaded into the data vault and all the cleansing and application of business rules takes place downstream i.e., out of DW. This means that Data Vault model is geared to be strictly a data warehouse layer, not as a data delivery layer which still requires physical or Virtual star schemas or cubes for Business Users or BI tools to access.

Bill Inmon in 2008 stated that the “Data Vault is the optimal approach for modeling the EDW in the DW2.0 framework.”

In Part 2 and 3, I am going to explain different components of Data Vault and it’s power with the help of some examples.  That will clearly explains why the Data Vault should be a preferred “flavor” for different businesses.

- Jyothi Kaparthi

To get at the data that most executives feel they need to make accurate decisions, many turn to the manual modification of existing reports, or the creation of their own “Pet” spreadsheet they use almost daily, or certainly many times a week. 

 In an update to a report cited last spring on this site, a September, 2009 Dartmouth University study suggests that the error rates in formulas on spreadsheets in their study were only .087% of all formulas they audited.  HOWEVER, these were in cases where the formula produced the WRONG RESULT, and actually resulted in 87% OF THE SPREADSHEETS REVIEWED having errors in which the spreadsheet then produced the wrong result. 

How good is good enough?  What if you could reproduce the “Pet” spreadsheet in a true Business Intelligence solution which would ensure that the data and results in the sheet were as solid as the data in your transactional systems in the first place?  How much does the wrong data or the wrong decision cost you, or your company?  I would argue that “good enough” might just be good enough, if you could ensure that the data was accurate, and mitigated the possibility of error, while increasing the timeliness of the information to the decision maker.  We have deployed such systems in a couple weeks’ time leveraging tools like SharePoint, Excel, and other software products that our customers already owned, and quickly delivered a system to our customer where we dramatically increased the accuracy of their information.  These solutions form the basis of our iterative approach to Business Intelligence.

********************

Recently I came across a quote by the legendary Chicago Sun-Times journalist Sydney Harris who observed, “Our dilemma is that we hate change and love it at the same time; what we really want is for things to remain the same, but get better.” Nothing like a little paradox to reboot the brain and inspire a new look at the same landscape.

We want things to stay the same – only get better. Ain’t that the truth? There’s a lot of comfort in things remaining the same. We know what to expect, we can predict our reactions, nothing is going to catch us off guard and cause us pain or make us look bad.

We gain a sense of security in things remaining the same, especially over a long period of time. It’s like a baseball hitter who spends hours hitting off a pitching machine. The speed and location of the ball are predictable, so eventually, no matter how fast the pitch comes in, the hitter can whack it. He starts feeling good about himself. Then he gets into a game where the pitcher is changing speeds and location, and suddenly those hard line drives turn into soft pop-ups and groundouts. Without the predictability of the machine, hitting becomes a much tougher job.

Change by definition upsets the status quo. Sometimes that’s good. Sometimes it’s not. If it makes things better, then we love it. But because we only know the outcome after the change occurs, we hate the prospect of it, mostly because we’re afraid of losing what we already have. That’s human nature.

While I was in the process of wrapping my brain around this concept I was given a copy of a recent article by Geoffrey Moore from the Harvard Business Review.¹ The article has proved very helpful in understanding the power of this paradox – stay the same only better.

The always-insightful Moore pointed out that there are three terms or time horizons we work in. Normally we deal in the short-term horizon and the long-term horizon. But according to Moore, there is also the overlooked, often borrowed from and always-misunderstood middle-term horizon, which ironically is the only place where innovative ideas can gain traction.

Eureka! Paradox solved – or at least given clarity.

We are very comfortable in the short term, getting instant gratification for our immediate needs – be it food (hence the proliferation of quick-service restaurants), receiving a thumbs-up for doing a good job, making a quick sale, or achieving our quarterly quota, etc. Hitting a short-term objective is satisfying, although getting there can be difficult. Still, the shorter the term we’re dealing with, the fewer chances there are for the rules, the environment or the assumptions we’re working under to change. As Harris points out, we would prefer it if it were a bit easier.

We are comfortable in long-term thinking about the future, designing new products and services, opening new geographic markets and starting new businesses because we apparently enjoy a degree of accountability that is, shall we say, more fluid at the edges. The future is ripe with possibility, riches and romance; or as they say in baseball – all teams look good in spring training.

The other comfort with the long term is that if changes do sneak up on us, we will have time to react to them. Changes that face a long-term outcome aren’t nearly as traumatic, giving us the opportunity to try different things, regain our equilibrium and return to a state of nonchange before we reach the day of reckoning. Things may have changed in truth, but they don’t feel like they did as much because we have time to assimilate the changes.

“Our dilemma is that we hate change and love it at the same time; what we really want is for things to remain the same, but get better.”

What Moore gets right in his essay and that the Harris quote misses is that innovation is not actually a dilemma. A dilemma is a choice between two painful alternatives. Moore demonstrates that there is a third alternative, a middle horizon or a middle term, that incorporates the best of the other two. He says that in order to implement change, we need to create a space in this middle horizon that is free from the pain and rewards of the short-term horizon and also free from the open-ended “explore all options” thinking of the long-term horizon.
 
What’s needed, then, in order to implement innovation and alignment in the middle horizon, is a little wiggle room. I realize the term “wiggle room” isn’t listed in the glossary of the latest MBA textbooks – but it works for me. Wiggle room means there is flexibility in the business expectations of ROI and market share for new product and service innovations, making the prospect of change a less fearsome one; but it also means there are needed restraints that sharpen the focus. There is less of a tendency to push off concerns about the consequences of your actions on “future you” when the future is not as far off. Flexibility with restraint is the ideal environment to nurture innovation.

One of the ways to create an innovative middle horizon is to build both flexibility and structure into an IT organization through the use of managed services. On the flexibility front, managed services give companies two critical advantages – flexibility of capital resources and flexibility of human resources.

Innovation by its nature requires a large investment in human resources in particular. In most cases, it helps to have many minds brainstorming a variety of concepts from different points of view to nurture innovation. It also takes a fair amount of freedom from the restraints of day-to-day work in order to envision what does not already exist or is not already a part of the corporate culture. Yet it is difficult to achieve that free-thinking mindset when your best resources are bogged down in the day-to-day tasks involved in keeping the current business operating. Offloading the mundane tasks onto a managed services provider frees your experts to think in an innovative way. Reducing current cost and avoiding future costs also enables new products and services to attain more realistic maturity cycles.

Managed services address the structure part of the equation by making the costs of ideas real. In a typical organization, the cost of the day-to-day running of IT tends to be loosely defined, coming out of a central budget that can be applied conveniently. When working with a managed services provider costs are much more tightly controlled, with greater accountability across the board. Knowing this level of detail helps place a practical focus onto innovation, assuring that it is being driven by the needs of the business, not just innovation for its own sake.
In the end, business survival is about bringing innovation to market. Managed services are an often-overlooked yet critical tool in a CIOs portfolio for creating that little bit of wiggle room in the mid-term horizon for new products and services to find their potential. They can help take some of the fear and pain out of change by redefining roles in order to encourage it. While the individuals may not learn to love change unequivocally, they may at least learn to embrace it as a necessary step on the road to success. And that’s definitely movement in the right direction.

Reference:

1. Geoffrey Moore. “To Succeed in the Long Term, Focus on the Middle Term.” Harvard Business Review. July/August 2007.

– JB

During the recent recall during the recent 4th of July holiday, no less, a local grocer notified their customers in a variety of ways. They placed signs in stores in meat departments (but if you are in the store buying meat – chances are you may have already consumed the meat you previously purchased).
Thanks to their frequent shopper program, they were able to track purchases by customers who use the company’s loyalty card. This is normally used to track purchases to accrue fuel discounts, or receive coupons and discounts on future purchases. During the recent recall, they printed a message at the bottom of the receipt to customers whose history showed they may have purchased the recalled product (and instructed the checkout clerk to point out the notice to consumers). In other cases, they actually called customers who used the loyalty card to purchase the tainted product (what a great motivating factor for encouraging consumers to keep their card information updated!).

Obviously a phone call would be best – since there is a time dependent factor for many shoppers – who purchase and immediately use the meat they purchase – but register tape alerts are great for those who stock up and placed the meat in their freezer for later consumption. Better late than never.
Loyalty programs sometimes come under fire – “what is being done with the data?”
“Are they selling personal information?” “Why do I have to give up that information to get lower prices?”

Generally, however most retailers promise that any proprietary information is not shared and only aggregated item data is shared with their vendors. In today’s competitive environment, businesses continually strive to “focus on the customer” and make the customer “# 1” – and develop the strategies that put the customer at the center of their business. Doing so makes customer data increasingly important, with loyalty programs serving as a very effective way of capturing customer data.

Another benefit of customer loyalty programs is the ability to foster two-way communications with customers – this can be done by implementing an active customer feedback system and listening to customers. It not only provides a way to capture customers’ concerns, questions and complaints, but as described above, it can offer a great way to reach out to customers.

For now, I’m fostering two-way communications with my customers by meeting with them, and having conversations with them about how they are communicating with their customers – both internal and external. I’m learning a lot, and hopefully gaining insight into how we can best work with them to facilitate this process. Clearly “one size fits all” won’t be the right answer. By listening, and asking questions, we can make a lot of progress in jointly defining the solution. How are you communicating with your customers?

It seems that “Taxonomy” was my word for the week. This is my third post about it within 7 days. It’s not that I am in love with the word, rather, it’s just pretty darn important! With any big initiative, the first thing we look to is a solid foundation for communication. Think about it, we usually address taxonomy anywhere from casual discussions to formal governance groups for many initiatives – dare I say any initiative that strives to bring real change to an organization begins with taxonomy (either consciously or subconsciously). Thinking it through, here are some of my top-of-mind game changers that require a solid taxonomy:

• Master data management. By definition this is really an enterprise taxonomy that is the official reference data for an organization.
• Metadata management. Tagging data with information is best performed only after a taxonomy is well-established. Else, with-what-shall-I-tag-it plagues the process.
• Business Intelligence. Without a proper taxonomy, how do we bring together people from diverse business perspectives together to understand data from a central and enterprise viewpoint?
• Data Management. Well, of course we can’t properly manage data without knowing where things are in a hierarchy or what context the data should exist in.
• Data Quality. Here we are really measuring data against the taxonomy; whether implicit or explicit.
• Governance. Strategic decisions are made for specific purposes and they need to rely and depend upon a socialized and accepted taxonomy.
• Data Stewardship. This is the process of holding someone accountable for making tactical decisions to implement strategic direction.
• BPM. When we look to manage business processes, they depend upon real information. So, having a taxonomy to base these data points is crucial.
• SOA. Reusing software components and exposing them at the enterprise level demands a highly accepted understanding of the organizations data. Sure, this view is exposed as a group of web services that are published in a repository that is self aware – but without a canonical data model as your underlying foundation, consistency is not reached. A canonical data model is highly correlated to a mature taxonomy.
• Strategy, Solutions and Architecture. It’s near impossible to calibrate these three without a friction free flow of communication. Let’s not talk about what should’a, could’a and would’a – but let’s focus on the business problem at hand. Having a living taxonomy that is socialized, accepted and part of our DNA is key to gaining quick momentum as we put distance between us and our competition.

These are just some quickies that I bubbled up. What other initiatives need a solid taxonomy? Thinking about taxonomy, when you look to bring real change to an organization, what happens? From my experience, there are two choices:

1. Address taxonomy early and often. Realize that there are some things that are so important that we need to establish, socialize and enforce them.
-OR-
2. Jump to build a solution. Then realizing there are terms misaligned, misused and duplicated, go back and either fix the data models (and all subsequent diagrams and code – this rarely gets done) or create a lot of code to hide these issues. When we do this, we establish a short term brittle foundation that breaks when the next change comes or we end up with a bunch of custom spaghetti that tries to tie things together but really ends with just a lot of confusion.

Bottom line
• Embrace taxonomy within your natural collaboration style. When something is unclear, pause, ask, record, check for understanding, agree on the outcome and move forward. It’s not a development phase, don’t sell it to senior management. No one cares about it. It’s an expected minimum of doing business. Add it to your culture’s DNA.
• Don’t underestimate the issues when terms are not aligned. It wrecks havoc to your foundational infrastructure and the costs (both hard and soft) can be big. Know it is there and plan for it.
• Scope your risk. If you are working within a group or team, the risk is small. Plan for it and cross it off your list as you develop it. However, if you are aligning silos or working across divisions or bringing others into alignment, or working cross-culturally, or introducing new teams, these issues can be big. Again, plan for it – put someone in charge of its care and feeding.
• Use it as a way to create excitement and ownership. Once you work together, it is always good to look for accomplishments to celebrate. Depending upon your scope, it can also be a way to generate a new level of buy-in. Manage the group right and they walk away with the justified feeling that they had part in it – that they created it and it reflects their slice of the business. Trust me here – then they will socialize it and ensure that its followed in their domain!

Now that is exciting stuff!

~ Scott Felten

As in anything of substance, look towards your ancestors! Before beginning new journeys, look at the travels and teachings of the ancients. Once upon a time, in a land far far away, things were very expensive and the littlest of changes translated into huge dollars. The ancients lived in those lands and had to navigate through the treacherous waters of high hardware costs, outrageous communication costs, high people costs, massive lines of expensive code and hidden dependencies. In this land, things were new and every notion of business had to be created. In this time, the people who inhabited that land had to use their brains, all the time. There was no drop and drag. There were no visual approaches and pre-established templates. Things had to be thought through in great detail and time was spent on foundational issues…because if we don’t do it right from the beginning, it will be bad, very bad.

So, pull out that old book in your company and turn to that portion that equates to Leviticus – read about your Moses, who came down from the mountain with the law…what was good, what was not. Listen to those stories of old and take stock. It is said that history repeats itself. Why not leverage what’s been done and what’s been done with a rigor that I would say is not the norm of today.

Turn around right now and look at the shelf behind you. There, in the corner, you see it? There’s a book (or two) right there – yes, it’s the 3” black binder that has dust all over it. Take it down, offer a quick thanksgiving and open it slowly. Spend some time with the ancients and understand where you came from – you may just stumble upon truths so great that they make your hair white. At the very least you will walk where the great ones walked and who knows maybe you can point the way to the Promised Land!

You never know where you will find gold – roll up your sleeves and start digging around!

~ Scott Felten

Then while he was 3 years old he spoke his first words. His mother told me this way. So, it was lunch time and ‘my son’ turned to me and said…now remember, this was his first words… “Mommy, may I have some lunch please”?  She was shocked and of course very relieved. At the next doctor’s visit, she told the specialist the story. After looking at all the facts, the doctor described things as… Well, you son is very bright and a perfectionist. He was living in a multi-cultural home where both English and Spanish was spoken. Before he chose to communicate (with words), he had to fully understand the framework of grammar and its nuances. Then, he had to work through the process of selecting his base language (will it be English or Spanish).

Fast forwarding the story a few years finds ‘their son’ in first grade. Upon completion of further testing, they found that he was in the top 99.999 percentile in math – among college aged men and women.

This story is not much different than some of the issues we face each day with our clients. More times than not, each team, group, department speaks a different language. I’ll never forget the time where I was asked to develop a strategy for a recognized revenue solution – where they wanted to better align revenue, cost and time to properly manage cash flow and reporting. I was working directly for the CFO and one of the requirements was a report that grouped data as follows, she called it ‘The Region Report’:

• New Business
• East Division
• West Division
• Canada
• P&G

The solution was an education on taxonomy. As mature as this company was, they did not operate with a known taxonomy. Much like my friend’s son choosing not to speak until he fully understood what the framework was and how a taxonomy carried made the context known to both sides, I worked to help define the common taxonomy. Once we did this, we were rocking. This application became on of the strongest points in the organization, because we built a foundational canonical model around a true taxonomy that we socialized. This really became the hub of all data points and drove both master data efforts as well as metadata disciplines.

Looking back, it seems obvious and if you did your job well, this should be the case. Can you guess as to the structure in my oversimplified snippet of a real issue?

The above was decomposed to:

• Country (is a parent of region)
• Region (belongs to Country and is a parent of SVP)
• Senior Vice President (works within a Region and is the parent of a VP)
• Vice President (works for a SVP and owns one or more clients)
• Client (is owned by one or more (many to many) VPs and has a status of Type)
• Type (describes the client and consists of either ‘New’ or ‘Established’)

Also, we have the notion of a date for transactions as well as a time-based dimension for SVP, VP, Client and Type – so people, clients and type can reflect ownership movements.

It might seem painful to go through the rigor and discipline of establishing and socializing a true taxonomy, but its worth it. It’s not much different than building a house on solid rock…why rush and build things on sand. Don’t fall for a ‘Con’ – make sure your text resides within the proper context!

Happy Building,

~ Scott

Or is it the ‘Magic’ of the wheel….

Communicating an idea efficiently is challenging, especially when that idea covers many IT functions that can each by themselves stand alone. In an effort to communicate our Business Intelligence and Data Management capabilities in a way that takes less than a 7 day training, we put together this graphic. Of course, the colors were selected by IT guys and not by our marketing department, but we feel that we have captured the essence of our complete capabilities within the BI/DM space – the stages and relationships of our capabilities as well as the breadth of our talent.

Internally, we call this model the FABI Wheel. Why FABI? It stands for the (Henry) Ford Approach to Business Intelligence, and was inspired by our smart guys (and gals) here at LUCRUM. This unique approach is important because it is an approach to thinking about business intelligence and data management in a way that is not driven by technology or bad habits. It is driven by the pursuit of delivering value quickly by unhiding data in a method that puts ownership in the hands of the business people. We know that BI is a journey, we have been there before and this is how we think. Let me explain…

First, the outer wheel has three sections; Strategic, Tactical and Operational. This communicates that LUCRUM has offerings in each of those areas. We have a mature strategy and can engage at these levels.

Strategic. This is the level that can set us apart from others. It is here that we help our customers set direction. If we engage here correctly, we will win a client for the long term. At this level, we are architects (helping to ‘plan the city’, where the city represents systems) – we build relationships and architect solutions here.

Tactical. This is where we are the builders. We deploy as warehouse architects and engage to carry out the plan. We are builders and implementers. Our unique collaborative and iterative approach sets apart from the rest. We focus all our energies on delivering value and doing it quickly!

Operational. This is where the results are seen. This is the 10% of the ice berg that is visible. We are deploying here, always listening and always understanding. We are working to bring the parts of the customer’s organization together – we are working as facilitators of change.

Second, the outer wheel is self-sustaining and self-connecting. It’s easy to see the transitions; but notice the transition from Operational to Strategic. Within the operational phase we are always listening and always understanding…so that we can complete the feedback loop and build ‘the next strategy’. As we deploy reports, cubes, dashboards, we are touching the area of the business that is strategic, they need the unhidden information to make the best decisions. Well, there is always that next round of unhidden information and it is here that we make relationships and bridge for the next opportunity – which naturally begins with strategy; we are either building out the strategy to the next level or developing game changing strategy with the customer. It’s how we provide extra value to our clients and mature our relationships.

Third, the inner wheel does have connections to each of the arrows both before and after. These connections are fuzzy, but real. As we operate within one arrow, we have direct opportunity to grow our business in both the next one and the previous one. For example, if we are building a warehouse, we will have visibility in to the next effort, reporting. But don’t forget the previous arrow’s opportunities ‘data integration’. If we get tasked with the data integration portion, don’t forget the previous opportunity ‘MDM and Metadata’. It’s not possible to integrate without a strategy for MDM and Metadata – how far we go with this depends upon our relationship with the customer. The magic of the inner wheel is to help us understand both where we are and where the opportunities come from (looking forwards and backwards at the same time) all to deliver value to the client so that they are successful.

Fourth, finally, putting it altogether, we are focused on getting the right information to the right people delivered the right way at the right time…so that our customers can engage their business and succeed. This is the heart of the wheel.

~Scott Felten