Better BI on Bigger Data
September 1, 2010
“Can we do BI without a significant infrastructure effort?”
What’s the most common BI tool? Microsoft EXCEL! If you are considering a DW initiative, you are likely encountering resistance from your hard-core, Excel based, analysts. Taking away Excel from them means that they will have less flexibility and spend more time waiting on changes. I’m guessing that they may not be on board with your vision.
1010Data is a zero-footprint, browser based data warehousing product. It has an Excel like feel, but it can handle billions of rows of data (not thousands like Excel). It’s fast, flexible and requires no infrastructure support from IT (if you are in IT, my guess is that you may have already stopped reading this article). The system is not an OLAP system, it requires no design, and not even indexing strategies – but it’s so fast!!
This is a new paradigm shift: from ETL to ELTAR – extract, load and transform as required. Hmmm….cleansing and summarization at query run time; views are used for data governance.
Don’t think it’s true? Dollar General is using the platform and was up and running in 5 weeks to 115 users with a 100% ROI.
What could you do for your customers if you could have their DW up in just a few weeks instead of a few months??
- Jodie
Making Information Available
August 9, 2010
I’m not sure if you’ve noticed, but I’ve not been blogging with the same gusto as of late. Ah the life of a Consultant.
I have been working with a local financial institution creating financial models this summer. (It leaves me with little time for blogging.) I did happen to stop by our 7755 Montgomery Road office today and checked my mailbox. In it was this month’s Information Management mag. I was immediately drawn to this month’s Snapshot: Making Information Available. Here’s some stats for you to consider:
61% of respondents are less than satisfied with their current process of creating information applications and are only lukewarm about their current information application technology. Here are their complaints:
- It takes too long to assemble and deploy applications.
- It is too difficult to assemble and view information into a simple view.
- There are not enough capabilities to integreate and normalize information from disparate applications.
WOW! I ask all of you fellow BI folks out there…what are you doing to solve this problem??? Why is it with all of the tools available today, our users are finding it too difficult to use them!! What are WE doing wrong?
As I mentioned, I am working with a customer on Financial Models this summer. I am fortunate to work with some SUPER SMART people in this group. They have come up with the most ingenious ways of getting their data out of old clunky systems. They can create some of the most INSANE Excel formulas to manipulate data! Their Excel sheets are visually appealing and get data to their management in a timely manner. I’ve had some spreadsheets that have taken me days to figure out the Excel formulas (and I’m a guru!). They are awaiting IT to “build them a DW” to make their lives easier. Here’s to hoping that it can deliver on their expectations! Here’s what I would do to ensure that it does:
1. Use an iterative methodology to build the DW. Recreate existing Excel reports from the DW as you go.
2. Implement a user-friendly reporting tool that allows them to create their own reporting. Give ‘em lots of drag and drop functionality and make sure it can Export to Excel.
3. Create a request process that allows the DW to change with the Business. Creating a process that queues up the work for months and months does not help the business user to create the financial package that’s needed at the end of the month.
4. Keep the model flexible. Doing this will ensure that you can always add a new organziation, hierarchy or measurement.
5. Build cubes! These users are smart cookies and they aren’t afraid of a Pivot Table. Give them the flexibility and performance of a cube and let them start to uncover their data.
Hmmm…what’s missing from my list? What would you add?
Happy building!
- Jodie
Business Intelligence Symposium May 6th!
April 6, 2010

Join us on Thursday, May 6, 2010 at the NKU METS Center for a half-day symposium of collaborative learning, focused on business intelligence. The Business Intelligence Symposium brings together regional business & IT executives to learn how their peers have been implementing data analytics, business intelligence solutions and Dashboarding. The emphasis of the symposium is to share ideas, stories, experiences, and business cards. Case studies, along with live demonstrations will be presented. Breakfast and lunch will be provided in a collaborative environment that facilitates peer networking and BI discussions for an enhanced learning experience.
View the agenda below and register today for $49 at the following link: http://tinyurl.com/yef3khh
Agenda:
7:30am – 8:00am Registration and Breakfast
8:00am – 9:00am David Holcomb, PhD – Director, Data Management, Western Union Simplicity and Transparency – How to do Effective Data Warehousing and Business Intelligence (Presentation)
9:00am -9:45am Mr. Steve Hangen – CIO, WinWholesale BI Roadmap – A Project, a Journey, a Culture (Presentation and Demo)
9:45am -10:00am Coffee Break & Conversations
10:00am – 10:45am Mr. John R. Ward – Director, Health Systems Integration, TriHealth The New Era of Healthcare Clinical Information Systems Unstructured Data – Internal/External
10:45am –11:30am Mr. Jeff Shaffer – Vice President of Legal Operations, Unifund Visualization – Running a business with Dashboards and Scorecards (Presentation and live Demo)
11:30am – 1:00pm Lunch /Panel Discussion led by Dr. David Holcomb and guest speakers
Using OLAP to Improve Organizational Effectiveness – Part 3
March 21, 2010
This is the third and final post in my series on using OLAP tools to improve the effectiveness of organizations. In Part 1 I discussed some background concepts and terminology. In Part 2, I talked about some specific examples of how OLAP can have an impact in this area. In this post, I’ll talk about a specific application: utilizing OLAP software to provide improved performance feedback to employees.
OLAP and Performance Feedback
Improvements to organizational effectiveness can also be realized by utilizing OLAP tools to provide performance feedback to individual employees. Improved performance feedback will help employees achieve group and individual performance objectives. Increased attainment of these individual and group performance objectives will, with proper alignment of these objectives and organizational objectives, improve organizational effectiveness.
There are several advantages to providing performance feedback with an OLAP tool. If the situation is right, feedback can be provided:
- At an individual level
- On a larger sample of employee activity
- Quickly
- In a meaningful manner.
Common Problems with Performance Feedback
Organizations often make attempts to improve the provision of feedback to employees. Newsletters with departmental performance numbers, posters in gathering places displaying performance charts, and managerial reports with quantitative measures of performance are all attempts to improve the distribution of feedback to employees throughout the organization. One problem with such efforts is that they are usually not provided at an individual level. Feedback on departmental, team, or group performance is certainly helpful but depending on the size of the group, its effect will be limited. Individual performance feedback has its own problem in that it is often time prohibitive to provide extensive individual performance feedback. The result is often weekly or monthly group performance feedback with individual feedback coming only during annual or quarterly reviews.
Individual performance reviews often suffer from another problem: small sample sizes for review. If an insurance company is reviewing the performance of claims adjusters using manually prepared data, it may be impossible to review more than a small sample of the adjuster’s work over what is typically a long review period. Small samples may, of course, result in a flawed appraisal of an employee’s overall performance.
The elapsed time between events reviewed and performance appraisals is also a problem with traditional feedback provision. Consider the timing of typical reviews: an employee makes a mistake in handling a situation in January, the incident turns up in a sample taken in May, and a review is finally conducted in June. If a review had been conducted immediately following the incident, the chance of the employee repeating the mistake will obviously be lower.
Traditional feedback provision often suffers from poor presentation of the message. An interview conducted by a busy manager attempting to perform a number of appraisals in addition to other work may not be optimally effective.
Performance Feedback Improvements with OLAP
Utilizing an OLAP tool may remedy some of the traditional problems with employee feedback. Imagine again the situation of an insurance company reviewing the performance of claims adjusters. As a solution to the problems listed above, an OLAP cube could be developed and made available to adjusters on a daily basis. Adjusters could be presented with individual performance feedback delivered via the web. They could see at a glance how their activity the previous day compared to group averages and organizational objectives. Exceptions could be noted immediately by the individual employee, rather than organizational objectives. Exceptions could be noted immediately by the individual employee, rather than a manager, and quickly corrected. Feedback could be provided on all activity from the previous day or week rather than on a small, dated sample. Finally, feedback could be presented in easy to understand charts which, in addition, roll-up to display departmental and organizational performance as well.
Improved performance feedback gives employees the ability to monitor their own performance and to take corrective action quickly. By improving the ability of individual employees to meet their performance objectives, the ability of the organization to meet its objectives and fulfill its mission is improved as well.
Conclusion
OLAP technology can improve organizational effectiveness by:
- Improving management’s knowledge of progress on objectives
- Improving employee coordination on efforts to achieve these objectives
- Communicating the link between employee effort and performance
- Communicating the link between employee performance and reward
- Improving employee performance feedback.
Although OLAP tools can provide assistance in these areas, their impact is obviously limited by factors specific to each organization. An OLAP tool cannot compensate for poor development of objectives, poor performance reward systems, or any of the other organizational factors discussed. Utilizing an OLAP tool as I’ve described in this series with no attention given to the underlying systems it is trying to address will, at best, have no effect.
In an organization that has clearly defined its objectives and has implemented well-designed reward systems, utilizing an OLAP tool as we’ve discussed can offer a tremendous payoff. The ability to provide employees with improved performance feedback and to demonstrate the link between individual performance and organizational performance is extremely valuable. By helping an organization align individual goals with corporate goals, an OLAP tool can help an organization become more effective.
Good enough?
March 16, 2010
When is good enough, well, good enough? I suppose that depends, one old argument says that close only works in horseshoes and hand grenades. Can it work with decision making? How about decision support systems? Is good enough the manually created spreadsheets that over 90% of organizations use for decision support? I would argue that while it’s not good enough, most business decision makers work that way.
To get at the data that most executives feel they need to make accurate decisions, many turn to the manual modification of existing reports, or the creation of their own “Pet” spreadsheet they use almost daily, or certainly many times a week.
In an update to a report cited last spring on this site, a September, 2009 Dartmouth University study suggests that the error rates in formulas on spreadsheets in their study were only .087% of all formulas they audited. HOWEVER, these were in cases where the formula produced the WRONG RESULT, and actually resulted in 87% OF THE SPREADSHEETS REVIEWED having errors in which the spreadsheet then produced the wrong result.
How good is good enough? What if you could reproduce the “Pet” spreadsheet in a true Business Intelligence solution which would ensure that the data and results in the sheet were as solid as the data in your transactional systems in the first place? How much does the wrong data or the wrong decision cost you, or your company? I would argue that “good enough” might just be good enough, if you could ensure that the data was accurate, and mitigated the possibility of error, while increasing the timeliness of the information to the decision maker. We have deployed such systems in a couple weeks’ time leveraging tools like SharePoint, Excel, and other software products that our customers already owned, and quickly delivered a system to our customer where we dramatically increased the accuracy of their information. These solutions form the basis of our iterative approach to Business Intelligence.
Using OLAP to Improve Organizational Effectiveness – Part 2
February 28, 2010
This is the second in my series of 3 posts on using OLAP tools to improve the effectiveness of organizations. In Part 1 I discussed some background concepts and terminology. In this part, we’ll talk about some specific examples of how OLAP can have an impact in this area.
OLAP’s Impact on Organizational Effectiveness
How can an OLAP tool help improve an organization’s performance as measured against its objectives? Answering this question requires a greater understanding of how strategies and tactics are implemented within organizations. I’ll use a model of organizational effectiveness developed by Michael Beer to illustrate the implementation of strategies and tactics.
The picture below shows a simplified version of a model of organizational effectiveness developed by Michael Beer (Note on Organizational Effectiveness, 10). Business goals and strategy influence and are influenced by top management. Management determines and implements the proper organizational design to achieve the organization’s goals. The design of the organization, in turn, influences human resources attributes of the organization. Finally, these HR attributes directly impact organizational effectiveness.

This simplified version of Michael Beer’s model is presented again below. Added to the model though, is the position of an OLAP tool in improving organizational effectiveness. OLAP technology exerts its influence on organizational effectiveness in three sections of the model:
- Management
- The Measurement and Reward Systems aspects of Organizational Design
- The Coordination aspects of Human Resources.

While the impact of OLAP technology in each of the areas above is slightly different, each is related and shares a common trait: improvement in communication. Utilizing OLAP tools to improve communication requires a broad audience for their utilization. OLAP tools are traditionally utilized by analysts and managers. In this model, front-line employees become critical users of the tool as well. The wide-scale availability of web-based OLAP tools makes such organization-wide implementations cost-effective.
- The expectation that effort will lead to performance
- The expectation that performance will lead to reward (Vecchio, 185).
- Total technical support calls
- Total calls requiring a call-back
- Total number of complaints
- Number of minutes to resolve a call
- Customer survey ratings of support representative performance.
- Their level of individual performance
- Their performance compared to targets and to organization averages.
New Partner: TARGIT!
February 22, 2010
Have you heard of TARGIT? TARGIT is a suite of BI Tools geared toward getting you to BI “in the fewest clicks”. LUCRUM has always been a big believer in doing BI..Faster! This suite of tools is a great tool in our toolbox. We encourage you to learn more: http://www.targit.com/Products/TARGIT_Suite.aspx
Using OLAP to Improve Organizational Effectiveness – Part 1
February 21, 2010
OLAP tools have been widely available for years and are in use in a large number of organizations. They are typically deployed as speedy, easy-to-navigate reporting tools. With a little creativity though, this class of software can also be utilized in a very different manner.
As organizations struggle to communicate their objectives to employees and to align the activities of those employees with the objectives of the organization, they can get help from these same OLAP products. OLAP software can help by providing the capability to:
- Improve management’s knowledge of progress on objectives
- Improve employee coordination on efforts to achieve objectives
- Communicate the link between employee effort and performance
- Communicate the link between employee performance and reward
- Improve employee performance feedback.
In this series of three posts, I’ll talk about the role OLAP tools can play in each of the areas above. But first, I’m going to start out with an introduction to the concept of Organizational Effectiveness. This introduction will give us a structure to frame the rest of the discussion.
I am not going to spend any time defining OLAP. If you’re interested, check here and here for some background and definitions.
Organizational Effectiveness Defined
Effectiveness is defined as simply having the intended outcome. In an organizational context, the intended outcome is the goal of the organization which is usually expressed in a mission statement. The Hierarchical Definition of Strategy provides a framework for defining and explaining these concepts and I am going to use it extensively in these posts.
Hierarchical Definition of Strategy
Explaining organizational effectiveness requires a discussion of business strategy and the Hierarchical Definition of Strategy provides a simple framework for this discussion. The Hierarchical Definition of Strategy is built on the concepts of Mission, Objectives, Strategies, and Tactics (Barney, 10). I’ve drawn a simple figure below to help explain this model:
An organization develops its objectives based on its mission while strategies and tactics provide specific details regarding the attainment of these objectives. In the Hierarchical model, the effectiveness of the organization can be determined by simply comparing actual performance to objectives. Michael Beer summarizes organizational effectiveness in this manner:
“An effective organization is one capable of implementing its strategy … A strategy is implemented effectively when people and groups in the organization work in a motivated, skilled, and coordinated manner on the appropriate tasks.” (Note on Organizational Effectiveness, 10)
In other words, the effectiveness of the organization is determined by its ability to achieve its objectives.
Hierarchical Definition of Strategy – Example
An example will help to clarify these concepts and make them a little more concrete. Dell Inc.’s Mission Statement is:
The high level nature of the statement, though necessary, makes it difficult for individual employees to apply it to their daily efforts. At the next level of the strategy hierarchy, Dell management has likely developed Objectives that will lead to the achievement of this mission. For instance, we can imagine that Dell has defined an objective to “Provide customer support with a customer approval rating of over 90%.” This supports their mission of “…delivering the best customer experience…” and provides employees with a tangible performance target.
The final two levels of the hierarchy are related to execution. Strategy is a means to accomplish an individual objective. Continuing with our imaginary Dell example, the strategy developed might be “Deliver the fastest, most accurate technical support in the industry.” This supports their objective in the sense that a firm delivering the fastest and most accurate technical support would very likely receive high approval ratings from customers. Tactics are execution oriented and exist at the lowest level of detail. In the Dell example, a tactic may be a requirement that all customer support personnel complete a certain set of technical and communication skill classes.
In the example developed above, Dell’s organizational effectiveness can be determined by comparing actual appraisals of their support services with their objective of a 90% approval rating.
Next Post…
Now that we’ve laid out some concepts and terms, we can move on to the heart of the discussion. In Part 2, I’ll dive into the details and talk about how utilization of an OLAP tool can help an organization become more effective.
The Value of Slowing Down: Go Slow to Go Fast!!
February 10, 2010
I once read about a Chinese mathematician who calculated complex scientific formulas by hand using a slide rule. He lamented the rising cadre of scientists who punched formulas into calculators and computers. Although they worked more quickly, the new generation of scientists often lost sight of the concepts behind the calculations. Without this fundamental understanding, the younger scientists often failed to grasp the significance of what they were doing or apply concepts in new ways to make new discoveries or effective designs.
This story parallels an area in Information Technology called “Business Intelligence.” Business Intelligence is also known as “Data Warehousing” and “Executive Information Systems” with dash boards or digital cockpits. The IT organization provides a rich repository of data for the business knowledge workers. Providing data has become so important; in addition, the tools leveraged have become more and more rich in functionality. And yet, the number of business users truly leveraging this kind of technology-oriented business information environment lags the productivity that the organization could receive. Simple questions like: who are my best customers and why? What’s my best product and what is its margin contribution? Why is my market share in a particular geography increasing where in another market it’s declining? How can I get my business results information faster so I can be more informed on the ever-changing aspects of the market? A user says, I can make a lot of informed decisions….how can I make even more of them instead of hire more decision-makers? The business and market questions go on and on and on.
As IT professionals, we are used to being held accountable to deadlines with ever changing resources and requirements. In the world of Decision-Making, as data warehousing managers, we often are rushing to meet these same deadlines. Often the deadlines and deliverables overshadow the underlying purpose for building the data warehouse. The good thing about bad times is that they force us to slow down and painstakingly evaluate what we are doing. So, although there are dark clouds ahead, there is a silver lining in the reality of our environment in having to do “more” with “less” resources.
Here are 3 tips to consider making your Data Warehousing environment even more “ready” for business decision-makers.
- Meet with the Business Decision-Makers frequently. I am suggesting that a weekly meeting at a minimum would be beneficial in order to review their data, listen carefully to understand what data they are really using, and what data they may be leaving behind. Is the data they are leaving behind the result of not understanding how to use the data, is the data no longer relevant to their decisions, or perhaps the data is too summarized or too detailed?
- Document the business flow of the data graphically using business terms, not technology metadata definitions. Distribute the business document to all business and IT users so that everyone really knows how the data is being used in the context of business. Too often, we revert to memorizing the technical definitions and only use them. We lose the business context and as new people join the data analysis, the true business definitions are lost.
- Proactively have discussions sponsored by IT with the Business Users about the cleanliness of the data and how IT is transforming the data. Show them the techniques that you are using to cleanse the data and transform it so that there’s a common repository of data that they can use. The more the Business Users understand what you do in context of the IT problem, the more they will provide their insight into how the data is most meaningful to use.
Chinese “Business Intelligence” Proverb: If you plan for one year, plant rice. If you plan for 10 years, plant trees. If you plan for 100 years, educate mankind.
Tomorrow’s Forecast
January 29, 2010
I am always looking for different ways to describe what LÛCRUM does. Sure there is the standard response of “LÛCRUM helps companies to turn data into useful and actionable information,” but that can be tough to visualize. Sometimes it helps to use more vivid and familiar examples of things to explain the services we offer. Think of the weather. If all of the important weather components were just structured data in table or spreadsheet, it might look like this:
So sure, I could find what I was looking for…”what’s the temp at 9AM?” It takes a pair of readers a few seconds, but it’s there. There are so many other data points, however. Is it getting warmer or colder? Is it going to rain today? Certainly the other data points are there that would help me to make the decision – relative humidity, cloud cover, wind speed – but I may need to consult the company metadata to understand what it all means and if those numbers mean it will be getting hotter or colder. THIS IS DATA. Your org has it…you’ve got to make sense of it.
What LÛCRUM does, is make this DATA meaningful. We like to call it Business Intelligence or Data Visualization. Simply stated, we take all of those data points and help you to make better business decisions (or in this case, help you to decide if you should wear a coat or bring your umbrella).
THIS
IS
BUSINESS
INTELLIGENCE!!
Taking lots of data and making it meaningful…yeah, that’ s what LÛCRUM does.
- Jodie







