Browse > Home /

Business Intelligence, Done Right

October 14, 2008

Business Intelligence efforts often fail, not because of the technicians, but because of the disjointed relationships we (IT) have with the business. We fear relationships. We think that our investments and communications are a waste of time because we have a tremendous amount of downward pressure to deliver – so we start quickly and build according to some document that we substitute as face time with our partner. To fix this problem, we need to take the best part of our building experience as ITers and put that together with the best part of relationship management – skills that we often don’t posses, but are vital to successfully delivering value and meeting our partners expectations.
{more on “The Right Way” below}

Kimbal’s methodology calls for the segregation of the three layers of BI and this is fantastic. There are in fact three separate BI tracks that can and should run concurrently. The three tracks are:
1.    Technical Architecture
2.    Data Architecture
3.    Application Architecture
After all, we need to make progress on the technical choices of architecture and tools. What is our environment like, what standard tools do we use, what is our technical strategy to frame and contain our BI effort?
At the same time, we need to start looking at our data. What data will we need to work with, what is it, how should it relate, where does it come from, how do we extract, transform and load it. These are considerations of the data architecture effort.
Again, at the same time, what level of innovation do we need to arrive at? How do we present this to the user and provide the desired interaction to help mine data to craft information.

Kimbal’s Methodology

This methodology is right on! It is triggered by and the scope is managed by the requirements. These requirements are derived from that box that is labeled “Program/Project Planning”.  But it’s not enough, it’s not there yet! In my mind, this is a bit limited and really only addresses the “Go” portion of the project – to build. All too often, BI projects fail because of thing that are ‘above’ or before the Go. Here is what I have found to be the differentiating factors.

Requirements should be built from Strategy. Why are we building this BI application? What are the important questions that need to be answered? What is the value that these answers bring; are we making things better or faster or cheaper? A well formulated strategy addresses these questions and should manage the deliverables. When we rely on a set of requirements to build a BI solution, we are abstracted from the strategy and this is not good. When this abstraction occurs, we are removed from the business value and are relegated as a commodity…simply build to this specification.
However, strategy is not born without a birthing process. The birthing process consists of a couple of elements. The first is an alignment or a focus at most senior levels. If we can’t align on the true aspects of the business, we are forced to work in a vacuum and can, at best produce a solution that is severely slanted. This slant often caters to the loudest voice or the biggest ego. Worse, our solution is then forever raised in a silo that contributes to expensive growth and maturity with a limited enterprise appeal. If and only if  we get this alignment, we are doomed to deliver value slices rather than holistic value.
This alignment is not enough. After we have a focus, we then have to dive deep within the organization to see what raw materials we have to work with. I refer to this phase as ‘Discovery’. Once we are aligned and focused on the business value, what do we have to work with? What data exists in what format and at what availability and quality?
To state this concisely, we have the following necessary phases that will lead us to success.
1.    Alignment – A common focus with enterprise level buy in.
2.    Discovery – An exploration to ascertain our resources and an evaluation to ensure it supports the common focus.
3.    Strategy – The actual strategy that is an outcome of “What do you want” + “What do we have”.
4.    Requirements – The plan of how to get there from here.

Now, we are ready to “Go” forth and multiply! We are ready to build! And we know what we are building (requirements) and how the effort will benefit the organization (strategy). We know what we really have to work with (discovery). And finally, we have those committed at the top who are committed at the deepest levels of the effort and will both support/defend the effort as well as ensure that its accepted and reused across the enterprise.

Putting it all together, let me suggest the following methodology…

The Right Way to do BI

Sometimes we are asked “Why are you just sitting there? Do something!” We need to rally and fight the urge to just do something. We need to change that around as ask “Why are you just doing something? Sit there” (and get focused, aligned, discover, produce strategy, birth requirements…then we can “Go” and do something excellent).

The difference between a commodity and a partner is the level of time you invest in the relationship.
Take the time to do it right and it will be celebrated!
The next series of blogs will outline the above components; alignment, discovery, strategy, requirements and the three team approach to building BI solutions. Feel free to shoot me some questions or thoughts you may have regarding these topics.
Enjoy the ride!
~ Scott Felten

Written by davidebowman · Filed Under Business Intelligence, Featured | 1 Comment 

BI Maturity: You can’t get there from here!

May 20, 2008

I spent last weekend fishing with my father, brother and nephews. Since they live in Connecticut and I in Ohio – we decided to meet half way – somewhere in “The Alleghenies and Her Valleys” to quote the brochure. While I usually rely on my trusty Magellan GPS, I had given it to my oldest daughter to borrow as she was driving south where the weather is warm. So that left me driving through the mountains and her valleys around midnight. Driving through a small town on a small road, looking for a very small park proved more than a challenge. Since I have three daughters and one wife, I have learned to swallow my pride and ask directions. I did, but on my walk in to the “Sheetz” gas station, I was thinking they might give me that response…”Well, you can’t get there from here – you gotta go somewhere else, then loop back around to get there.” However, I received perfect directions! Thank you Mr. Sheetz!

But that got me thinking as I stumbled upon a poster from the TDWI folks that spoke about BI maturity and adoption. It was a few years old, but some things are so true. I was excited to see that old friend and took some time to write up my thoughts on the matter – as well as capture their context. You can visit their poster ’2005 BI Maturity Model’ at http://www.tdwi.org/publications/display.aspx?id=7288.

In case you are wondering how I am going to tie the first two paragraphs together, here goes. From my experience, when senior leadership learns of the value that BI can bring, they really want to ‘get this thing going’. They want to launch a large, comprehensive, enterprise-wide, based-on-the-new-tools, BI innovation that will hit home runs and win ball games. Well the problem is that you can’t get there from here. :)   There is a progression that must happen. Farmers can’t just throw seed on the ground and expect to make a profit - there is work to be done to prepare the soil correctly, then lots of care and feeding, praying for rain (but not too much rain) and then F  I  N  A  L  L  Y  –> the harvest!

TDWI states that “Most organizations go through six stages when evolving their BI environment from a cost-center operation to a strategic resource that drives the business and shapes the market.”

Using their framework, here is how I break it down…

1. The Beginning (TDWI calls it ‘Prenatal’): Since this is mostly financial, there are a medium amount of standards and not much flexibility. The control is dictated from finance or finance needs. Causal users account for most activity. Power users may make use of this type of information and leverage it into their own ‘shadow’ systems. The problem is that there is a large IT backlog for these reports. The problem here is the information gap – we get the information after the decision had to be made. This decision latency could contribute to the wrong direction as the data it is built on is often not fresh. However, at this stage, a general ‘awareness’ exists – there is at least the existence of the correct information. To get here, there is a larger initial investment and costs are high as economies of scale are not yet a reality.

  • Architecture: Management Reporting
  • Scope: System
  • Type of System: Financial
  • Analytics: Paper Report
  • User: All
  • BI Focus: What Happened?
  • Executive Perception: Cost Center

2. Army of One (TDWI calls it ‘Infant’): Lots of flexibility and not any standards to speak of – other than what is negotiated from one user to another. People think local and resist any global initiatives. Causal users use decline, while power users step in to take advantage of this new information. Still, power users are reliant on IT to set the stage for their data and IT continues to struggle with a backlog of requests. As we extract the right data and manually assemble it to address business problems, there is an understanding of what factors are leading to what business results. Cost are somewhat low as analysts are using their own tools and working with certain data sets extracted by IT.

  • Architecture: Spreadmarts
  • Scope: Individual
  • Type of System: Executive
  • Analytics: Briefing Book
  • User: Analyst
  • BI Focus: What Will Happen?
  • Executive Perception: Inform Executive

3. Working as a team (TDWI calls it ‘Child’): Flexibility is somewhat high, but at this point is waning as people within the department begin to work together. But still not many standards to speak of – other than what is negotiated from one user to another or driven from within the department. People still think/act from a local perspective and resist any global initiatives. Causal users use starts to trend up to take advantage of some individual benefits provided to them at the department level. Once the organization deploys data marts based on the emerging standards, the BI environment becomes a self service type, where the bottle neck that once existed within IT has been removed. At this stage, an understanding of why things have happened is occurring because knowledge workers are using analytical systems to extract data to their own needs and using that data to draw conclusions about business events. Costs begin to creep a small amount as some technology is purchased, but overall not a big factor.

  • Architecture: Data Marts
  • Scope: Departmental
  • Type of System: Analytical
  • Analytics: Interactive Report
  • User: Knowledge Worker
  • BI Focus: Why Did It Happen?
  • Executive Perception: Empowers Workers

4. Thinking bigger (TDWI calls it ‘Teenager’): Flexibility starts to fade as division wide standards arise. People see the need to work together and are driven by common divisional goals. Here there is an atmosphere of negotiation and consolidation as these standards are built out. Now causal user use is on the rise, as the wide standards lead to increased reliability on the data available within the BI ‘system’. Power users use remains flat; but their ideas are rolled back in to divisional solutions; they are seen as subject matter experts and are often tapped to provide leadership and direction for their domain.  The focus here is customized delivery at the divisional level; dashboards, scorecards, report cards and the like.  At this stage, managers are making use of the divisional wide dashboards and are given real time information that is actionable – what is happening right now.  Costs are rising as we work within the division to develop standards and customized delivery.

  • Architecture: Data Warehouse
  • Scope: Division
  • Type of System: Monitoring
  • Analytics: Dashboard
  • User: Manager
  • BI Focus: What Is Happening?
  • Executive Perception: Monitor Processes

5. Mature (TDWI calls it ‘Adult’): Standards are formed at the enterprise level. Governance groups are now formal processes with the proper structure and sponsorship. Senior level support is solid. Although flexibility takes a dip at first as the organization learns, flexibility then trends up as efficiencies and learnings are gained. Truly people are planning globally to act local at this stage. At this stage, executives are also onboard as the mature BI environment serves to align all players within the organization.  The causal users use the system to help them understand what they should be working on and how their efforts affect the organization as a whole. The delivery mode transitions from a divisional perspective to the enterprise but is also balanced; balanced or cascading scorecards are the focus of the organization and serves as the single point of truth to answer the questions about our goals and progress towards them.  At this stage, executives are using the BI environment as a communication tool to both align the organization on goals and objectives as well as communicate the results and current situations. These pivotal points are balanced all the way through the organization and are socialized in a manner that is equal to business strategy. The balanced or cascading scorecards open up alternative decisions as all indicators that lead to a ‘score’ are actionable. Costs rise again because of the enterprise level investment and collaboration, but the value should increase dramatically.

  • Architecture: Enterprise Data Warehouse
  • Scope: Enterprise
  • Type of System: Strategic
  • Analytics: Cascading Scorecards
  • User: Executive
  • BI Focus: What Should We Do?
  • Executive Perception: Drive The Business

6. Harvesting Relationships or Partnerships (TDWI calls it ‘Sage’): Leveraging the mature BI environment  by opening up that business service to clients is the last stage. Here standards and control continue to be formed, but originates from client relationships. Flexibility is also harvested as new ideas, thoughts, concepts are embraced by the mature BI environment. At this stage, our BI environment can now be thought of as a BI Utility for our customers; helping them to solve their business problems by making use of the organization’s rich and focused information in the form of customer focused solutions deployed to help build or bind relationships, thus increasing the value proposition to them by the organization. At this stage, the BI Utility becomes a needed part within the customer’s infrastructure (whether it be for a single customer’s need for specific and unique information or for a company with complex business processes). Costs take a dip, the data/information infrastructure is in place and the capital expenditure has been amortized. Tools already exist to develop rich applications.  The value increases exceedingly abundantly here, as a very narrow scoped application brings a deep penetration of partnership.

  • Architecture: Analytical Services
  • Scope: Inter-Enterprise
  • Type of System: Business Service
  • Analytics: Embedded BI
  • User: Customer
  • BI Focus: What Can We Offer?
  • Executive Perception: Drive The Market

Now that you know… Where do you see your organization? How can you actualize the next stage? What are some value statements that you can take to senior leadership? What business problems can you address? What type of socialization strategy will work best? Where should you invest and what will the return look like? Who can you trust to help you get there from here without shortcutting the maturity journey – proper growth is built on a series of solid foundations. These sucesses are the underpinnings of the needed BI elements; Trust, Vision, Focus, Value, Momentum…repeat.

Happy Maturing!

Scott Felten

Written by Scott Felten · Filed Under Business Intelligence, Technology | 1 Comment 

Oracle supports Microsoft

May 16, 2008

I can’t tell you how many times I’ve been in conversations around the topic of “Oracle vs. Microsoft”. I’ve heard both sides of the story ranging from “SQL Server for mission critical operations…are you crazy!” to “Oracle costs me my first born child…year after year!”. While these discussions are often entertaining, the line delineating the two database giants is blurring by each subsequent release.

In my years consulting for LÛCRUM, I have worked for numerous clients that have had installations of both Oracle and Microsoft running in their environments. With recent statistics estimating that Oracle controls >50% of the database market and Microsoft controlling >50% of the server operating system market, are you surprised? SQL Server only runs on Microsoft. Oracle offers more operating system versatility. While you’ll see UNIX and Linux installations, Oracle’s ability to run on Microsoft remains strong and they are improving their functionality with respect to Microsoft development. Where might an Oracle database deployed on a Microsoft server make most sense? In the small and mid-sized business market (SMB). In the SMB market, Oracle has competitively priced versions such as Oracle Database Standard Edition and Standard Edition One.

So what advantages does running Oracle on Microsoft have to offer? First, Oracle has tight integration with Active Directory and Windows Security Framework. Items such as single sign-on and security via database role and Active Directory group fall into this category. Next, Oracle offers 32-bit and 64-bit versions. In the 32-bit version, Oracle is able to utilize up to 3GB (out of a 4GB O.S. maximum) of system memory for database use. Finally, Oracle has also been working on enhancing its ability to integrate with the Windows development suite, specifically Visual Studio 2008. Oracle supports .NET in 3 ways. The Oracle Data Provider for .NET leverages ADO.NET API and allows .NET applications to access Oracle data. These APIs should be familiar to most Microsoft developers. In addition, through an add-in (free for that matter), developers can work with Oracle services via Visual Studio 2005 (and 2008 as previously mentioned). Through the development suite, developers have access to various wizards to perform various database tasks (i.e. DDL), a procedure editor (for PL/SQL procedures, packages, and functions), a Debugger for runtime error interaction, and integrated help for items such as Oracle error reference, SQL, and PL/SQL user manuals. Lastly, Oracle has integrated .NET extensions directly inside the database. This allows developers to created stored procedures and functions using C# or VB.NET within Visual Studio. This code can then be deployed to the database and referenced wherever a stored procedure or function is permitted.

Oracle has shown it is advantageous to offer solutions that fit neatly into an operating system that controls the majority of the server market, even if that vendor also happens to be a major competitor in the database market. Offer a product that is extensible and easy to use with development GUIs is sure to give you a seat at the table when it comes to choosing a solution for your organization. That is precisely why Oracle supports Microsoft (most of the time <grin>).

Dave

Written by Dave Cordas · Filed Under Business Intelligence, Technology | 2 Comments 

Get a handle on Unstructured Data

May 8, 2008

One of the big topics in data management these days is Unstructured Data.  What is it?  Word documents, spreadsheets, video, images, email, and instant messaging are a few examples.  How does one harness the wealth of information contained in these non-standardized formats, IF you are trying to capitalize on your existing data management infrastructure?  Microsoft has attempted to answer this question with its upcoming release of SQL Server 2008 (SS2008). 

Due out later this year, SS2008 provides built-in support for Unstructured Data through the FILESTREAM functionality.  FILESTREAM combines the power of a relational database platform with the storage flexibility of a NTFS file system.  This is accomplished by storing references within the database to binary large object data (BLOBs) residing on the file system.  In this fashion, SS2008 manages access and interaction with the information, but is not responsible for the direct storage of it.  Unstructured Data can be accessed through typical Transact-SQL statements or via Win32 API calls.  FILESTREAM is a good option to consider when objects being stored are larger than 1 MB in size and is limited only by the volume size of the underlying file system.  If objects are <1 MB on average, you’ll get better performance by using the Varbinary(max) data type directly within the database.

From a security standpoint, FILESTREAM fits neatly into the database.  If a user has permission to query a table and column containing FILESTREAM data, they are able to access the Unstructured Data.  This access however does not carry forward at the file system level.  Only the account running the SQL Server service account has access to the files at the file system level.    

Is this only way to deal with Unstructured Data?  Of course not, but it is an option.  There are some limitations when using FILESTREAM with other SS2008 functionality.  Special consideration needs to be addressed when utilizing Database Snapshots, Mirroring, Replication, Log Shipping, and Clustering.

Continue to browse through other blogs on www.thefuturevalueofbusiness.com to see conversations on SharePoint 2007 and its role in taming Unstructured Data.

Dave

Written by Dave Cordas · Filed Under Business Intelligence, Technology | Leave a Comment 

Dashboards for Dentists

May 1, 2008

This week I had my bi-yearly checkup with my dentist. I have never really minded going to these appointments. For the most part, other than always being gently scolded for not flossing regularly (come on, it’s a pain to do!), my visits are routine and without surprises. Over the last 10 years or so, one item I have always puzzled about is the WALL of patient files that exist behind the receptionist’s desk. How many patients’ records exist in that wall? How many are active patients? How many versions of x-rays exist per folder? What happens if the office goes up in smoke or is the victim of water damage? Does the history of the patient disappear?

Well this visit was a bit different, let me explain. I sat down in the chair and my hygienist explained it was time for x-rays. Ok no problem…put on lead vest…open wide and bite down on film…absorb some radiation…print out x-ray…review x-ray on white screen. To my surprise, my assumed process stopped at “absorb some radiation”. Instead of printing out the x-ray to film, the results of my x-ray immediately displayed on a LCD monitor next to my chair within “Dave’s dashboard”! My dentist had recently installed a new system and was in the process of converting the WALL into the digital age.

So what did “Dave’s dashboard” include?

  • A repository of x-ray films. This allows the dentist to quickly move between versions of films and allows him to monitor changing patterns in the mouth (tooth gaps widening/shrinking, jaw alignment, etc.)
  • Historical view of past visits. Included were procedures performed, costs associated, insurance company billed, future scheduled visits, etc.
  • A graphical representation of all the teeth in MY mouth. It showed my incisors, my molars, etc. But the neat part was that if something was “special” about a tooth, that “special” was represented in the graphic. Fillings were shaded grey. Cosmetic work was green. Areas “being watched” were blue. I was told areas with potential enamel problems would be another color.
  • A bunch of other “tabs” of information that I wasn’t able to view (couldn’t ask, mouth was full)

How cool is that? All information about a patient was online and accessible in a concise easy to read format. A format that can be shared directly with the patient to help them understand why their tooth is hurting or what their teeth may look like “after” a cosmetic change.

I attempted to ask my dentist about the software to get a feel for what technologies were used to create this. Obviously he didn’t have a clue, but it was running in Windows Vista and appeared to be client/server in nature. Not only did this appear to make the dentist and hygienist’s job easier from a paperwork perspective, but now this vital information was easily maintained, backed up and tucked away in the event of an office disaster. The reliance on the WALL was subsiding. So in a world where businesses are consistently trying to improve themselves, exposing the same old information in new exciting ways may just help turn on a light bulb to a new way of thinking or acting/reacting…even a dentist.

If I ever need to change dentists, I now only have to request my records be forwarded electronically to my next dentist. I don’t plan on doing so anytime soon though. I very pleased with my current one and he’s getting hip with the times and using software/technologies that I have expressed interests in.

Now where is that floss…

Dave

Written by Dave Cordas · Filed Under Business Intelligence, Featured, Technology | 1 Comment 

Next Generation of the Data Warehouse (DW 2.0)

April 10, 2008

I had coffee today with one of the authors of the new book, “DW 2.0: The Architecture for the Next Generation of Data Warehousing”, Derek Strauss.  Derek worked with William (Bill) H. Inmon and Genia Neushloss to write the book and it will be available July of this year (’08).  His company (Gavroshe) becomes the first independent consulting company certified by Bill Inmon as DW 2.0 Architects.  We talked about why DW 2.0 and what makes it different and he had some really interesting comments.

First of all, there is that big statistic that Gartner has reported…“50% of Data Warehousing will have limited acceptance or be considered failures.”  Derek commented that it’s frustrating to be in that area and to hear that statistic bantered around all the time. He challenges the premise…what is meant by the term ‘data warehouse’. After all, one man’s ODS is another’s datamart is another’s warehouse is another’s database is another’s xls sheet containing GL entries.  So, by setting a standard (DW2.0), it sets the baseline; OK now we know what a data warehouse is.

Second, they wanted to bring corrections to the DW environment. There are real issues around the traditional notion of a data warehouse. We talked about two of them at length.

Data currency – lifecycle of Data. Guess what, data has a lifecycle. Who knew? Well some of us experienced folks have known this for a while – but within the DW2.0, this notion is has a separation of concerns (to steal a SOA term).  Think about what happens over the years as data decays.  The use of an attribute changes over time as the business’ competitive landscape changes and they respond.  What happens to that data – do we declare a new attribute at some point, do we change it and push the new meaning upstream, do we just live with it?  One of the key pillars (or sectors to use Inmon’s terminology) of DW 2.0 is that the lifecycle of data is embraced.  Here is how it nets out… Refer to Figure 1 at the end of this entry.

 

  • Interactive Sector- The place where high performance data warehouse processing occurs (Very current data – Transaction data)
  • Integrated Sector – The place where integrated data resides (Current data – Relative to the business needs; hourly, daily, etc…)
  • Near Line Sector – The place where data with a lower probability of access resides (Less than current data – Also relative to the business needs; weekly, monthly, quarterly, etc…)
  • Archival Sector- The place where data with a truly low probability of access resides (Older data – We all know what goes here J)

Unstructured data – emails, documents, faxes, etc…  Traditional DWs had zero notion of unstructured data. A good analogy is the communications that occur between two people. The experts say that 80% of that communication is non-verbal and only 20% consists of the actual words we say (fun topic for another time). The same is true with data. For example…a CRM promises to give you a 360 degree view of your customer. Great, except that they forgot about the communications that we had with the customer. It’s like saying that the complete view is made with only 20% of the data.  The DW 2.0 architecture makes use of ‘probabilistic matching’ when looking at relationships between unstructured and unstructured data as well as between structured and unstructured data. Now you have a 360 degree view of you customer!

Enterprise Metadata Repository.  We did not talk much about the metadata aspect because no one really likes talking about metadata J. But, threaded throughout the architecture is the notion of metadata that has enterprise reach and natural to the data warehouse – a tight coupling.

In looking at DW 2.0, it is exciting because of the reasons outlined above. It was a struggle on several levels to have ‘one warehouse’ with no notion of data lifecycle.  On one level, it’s hard to treat data equally, when you really know that some stuff should be retired and some is volatile because it was just born.  To know that the teenaged data that behaves badly is managed in the same place, the same way as the 20/30 something data that is more mature. Also, when dealing with the finance folks, it’s easier to speak their language. No need to request money for another ODS when we already have a data warehouse…rather, we need to address the data lifecycle…data depreciation – or planning ahead; data accrual!  Once they understand that data is an assets and that assets depreciated or have a lifecycle, we can engage in conversations about how we need to manage this condition.

Of course, since DW 2.0 has separation of concern around currency – we can employ the technology that makes the best sense for that stage of the data (Oracle to handle the Interactive Sector, SQL Server to both the Integrated and Near Line Sectors, and Archiving on DB2 – just an example.)

Then leveraging emerging investments in MOSS becomes strategic to the warehouse (fodder for another day).

Lastly, since DW 2.0 has been trademarked, there will be no mistake about its composition.  Only the original authors and architects will be able to make a change.  Given the track record of these folks, I think that is a good idea. For more information on DW 2.0 visit http://inmoncif.com/home/.

In summary, DW 2.0 Advantages…

  • Hold data at the lowest detail,
  • Hold data to infinity (or at least to your retirement),
  • Not cost huge amounts of money,
  • Have integrity of data and still have online high-performance transaction processing,
  • Link structured data and unstructured data,
  • Tightly couple metadata to the data warehouse environment,
  • Support different kinds of processing without sacrificing response time, and
  • Support changes of data over time.

Take Care,

Scott Felten 2.0

 

 

 

 Figure 1 – DW 2.0 Architecture

 

 

 

 

 

 

Written by Scott Felten · Filed Under Business Intelligence | 2 Comments 

« Previous Page