Data Mining – We want it Now!
September 1, 2010
“We are drowning in information but starving for knowledge.”
What do you really understand about data mining? Dr. Candace Gunnarsson was formerly a professor of statistics at Xavier University. Her view of data mining is all about getting information out of the systems you have today and predicting future results. Her view is also about needing a DW modeled in a way that makes the mining experience easier and more meaningful.
Her view is that there is prescriptive (automated) data mining and descriptive data mining. I think most people think of data mining to be an automated process. Truly, the manual exploration must happen first. What do I think the drivers should be? If it’s predicting a purchase, I need to understand you as a customer. What magazines do you read? How old are you? Male/female? How often do you come to my store? All of these become data points in an algorithm that will lead to prescriptive data mining. If I can predict that you will make a purchase, I then need to be able to test my theory and provide results back to my model. Data Mining is Avery iterative process.
To have truly effective data mining, you need to have a multi-disciplinary team. Be sure to bring in your IT, Marketing, Finance, and operations-focused team members. They will all have different views of your customers and will understand their transactional data better than anyone else. Use all of these views to create a better view.
BIG Question – can you start data mining before you have your data organized in a DW?
I’d be interested in hearing your data mining stories.
- Jodie
The Science of Visual Analytics
September 1, 2010
Today LUCRUM hosted our second BI Symposium. Once again, it was well attended and we had some great speakers! I’m hoping that this becomes a regular event. If you have yet to attend, I encourage you to come to our next event (to be scheduled).
Our first speaker was Mr. Stuart Woodward, President OD OcuCue. (http://ocucue.com/) OcuCue is an interesting start-up that’s all focused on data visualization. I always love listening to visual experts. There is such a science to visual design. It’s about understanding the psychology of how users think and perceive what they see. If you are creating a dashboard, you have to design it in the way people think – we read from left to right, heavy color should be at the lower left hand side, etc.
“Good design has two key elements. Graphical elegance is often found in the simplicity of design and complexity of data.” – Edward Tufte
Mr. Woodward’s company creates meaningful dashboards that are icon based. They go beyond speedometers and graphs and actually create a customized dashboards with icons that are meaningful to the company using them. One example that he showed was for a hospital. There are some rooms that can only take female patients or only male patients. To show bed availability, their dashboard has a pink pillow or a blue pillow to represent which rooms are available. Hmmm…never thought of that!
How are you presenting data to your users? Are you simplifying the message? Setting up the information from left to right? Are your colors meaningful? (ie Red should mean bad, green is good)
OK…gotta run and listen to the next speaker!
- Jodie
Making Information Available
August 9, 2010
I’m not sure if you’ve noticed, but I’ve not been blogging with the same gusto as of late. Ah the life of a Consultant. 
I have been working with a local financial institution creating financial models this summer. (It leaves me with little time for blogging.) I did happen to stop by our 7755 Montgomery Road office today and checked my mailbox. In it was this month’s Information Management mag. I was immediately drawn to this month’s Snapshot: Making Information Available. Here’s some stats for you to consider:
61% of respondents are less than satisfied with their current process of creating information applications and are only lukewarm about their current information application technology. Here are their complaints:
- It takes too long to assemble and deploy applications.
- It is too difficult to assemble and view information into a simple view.
- There are not enough capabilities to integreate and normalize information from disparate applications.
WOW! I ask all of you fellow BI folks out there…what are you doing to solve this problem??? Why is it with all of the tools available today, our users are finding it too difficult to use them!! What are WE doing wrong?
As I mentioned, I am working with a customer on Financial Models this summer. I am fortunate to work with some SUPER SMART people in this group. They have come up with the most ingenious ways of getting their data out of old clunky systems. They can create some of the most INSANE Excel formulas to manipulate data! Their Excel sheets are visually appealing and get data to their management in a timely manner. I’ve had some spreadsheets that have taken me days to figure out the Excel formulas (and I’m a guru!). They are awaiting IT to “build them a DW” to make their lives easier. Here’s to hoping that it can deliver on their expectations! Here’s what I would do to ensure that it does:
1. Use an iterative methodology to build the DW. Recreate existing Excel reports from the DW as you go.
2. Implement a user-friendly reporting tool that allows them to create their own reporting. Give ‘em lots of drag and drop functionality and make sure it can Export to Excel.
3. Create a request process that allows the DW to change with the Business. Creating a process that queues up the work for months and months does not help the business user to create the financial package that’s needed at the end of the month.
4. Keep the model flexible. Doing this will ensure that you can always add a new organziation, hierarchy or measurement.
5. Build cubes! These users are smart cookies and they aren’t afraid of a Pivot Table. Give them the flexibility and performance of a cube and let them start to uncover their data.
Hmmm…what’s missing from my list? What would you add?
Happy building!
- Jodie
Current Openings with LUCRUM INC
August 4, 2010
Internal LUCRUM positions:
Contact Patrick Ryan at 513-564-6979 or send your resume to pryan AT lucruminc DOT com
Database Lead
SQL Reporting, SSIS, SSRS, Data Warehouse, Requirements gathering, Customer Facing, MS SQL Server or Oracle or Business Objects or Hyperion, some Arch exp, set a vision, Project Management Skills
Database Developer
SQL Reporting, SSIS, SSRS, Data Warehouse, Requirements gathering, Customer Facing, MS SQL Server or Oracle or Business Objects or Hyperion
Application Developer
ASP.NET, AJAX, Jquery & SQL (SSRS & SSIS)
Test Lead
Client Driven Development testing background, Automated Testing Tools
Tester
Client Driven Development testing background, Manual Testing
DB2 DBA
Lead/Specialist is viewed as an expert in the administration and support of database technology. Undertakes complex projects requiring additional technical knowledge and makes decisions on ambiguous administrative and support issues. Communicates common goals and direction for the team. May interface with management to provide project updates.
The position is on the Technical/Professional ladder at level 4 in the Database Administration discipline.
DUTIES & RESPONSIBILITIES:
1. Identifies, tests and deploys all database technologies and support tools (detailed knowledge of SMP/E, JCL, TSO,/ISPF, SQL).
2. Ensures system improvements are successfully implemented.
3. Responsible for verifying all data to be entered into database meets set standards and requirements.
4. Designs, enters, audits and maintains large, complex databases.
5. Leads others to identify, test and resolve complex database performance issues (e.g., monitoring and tuning).
6. Performs database system management functions (e.g., software installs, version upgrades and configuration management).
7. Defines and develops database standards, procedures and architecture.
8. Designs data models and DW database designs.
9. Facilitates project planning by establishing work plans, estimates, milestones and schedules.
10. Provides team with technical training and/or mentoring.
11. Other duties as required.
Application Architect
Overview:
Experience in application performance, troubleshooting, and analysis of java applications.
Experience in agile software development practices, continuous integration, and test driven development.
Experience in working with application profiling and performance monitoring toolsets.
Experience in working in a Services Oriented Architecture (SOA) environment and web services.
7-9 years object-oriented design and development of n-tier applications experience in multiple languages and related technologies (JAVA/J2EE, C++, .NET, Web Services).
Firm understanding of Web Service concepts and Internet technologies, WSDL, XSD, XML.
Must have excellent problem solving skills including troubleshooting and root cause analysis for large-scale, enterprise Java applications in an SOA environment.
Target Technical Skills:
Senior-level Java development skills: (required).
Open Source Frameworks: Spring, Hibernate, MyFaces/Java Server Faces: (highly desirable)
Application Server Configuration and Tuning: Apache, Tomcat, JVM tuning. (required)
Performance Analysis and Troubleshooting: CA Wily Introscope, YourKit code profiling: (experience with similar tools required. specific tools desirable)
Continuous Integration and Build Mgmt: Hudson, Maven, Ant, Junit: (experience with similar tools desirable)
Database: Oracle: (relational DB skills required. Oracle desired)
Servers: Linux (Redhat) and Windows-based. (Proficiency with command line tools/troubleshooting required)
Responsibilities:
Deep-dive Performance Analysis and Tuning
Troubleshooting for application exceptions at run-time and post-deployment validation.
Performs root-cause analysis and recommends solutions for application, environment, and performance-related issues
Build and Deployment Automation
Other development and application architecture tasks as assigned.
Release Manager
Overview:
This position will be responsible for managing the code that is moved from one environment to the next. They work closely with development and DBAs in order to thoroughly document and manage code as it moves toward Production. This position will be a point person for all code moving toward and into the Production environment. This individual will need to be a self motivated and collaborative IT Professional with a strong background in SDLC, software development and a working understanding of change and release management processes.
Responsibilities:
Responsibilities:
• Responsible for providing leadership and guidance to IT groups across the organization on Configuration and Change Management processes and procedures.
• Responsible for implementing and maintaining processes and standards to ensure organizational compliance with defined controls in the area of Application Change Management.
• Evaluates, selects and administers organization-wide automated tools pertaining to Configuration and Change Control and Build and Release Management.
• Implements Configuration and Change Management practices, standards and tools for use in development and maintenance projects.
Additional Duties:
• Develop and document instructions and guidelines to perform the functions assigned.
• Manage, maintain and deploy to test environments.
• Establish standard configuration management processes and procedures for version control, build and release management, CM audit reports, configuration identification and control, software product baselines and change management.
• Develop and manage systems of recovery for the code sources of the current and previous versions of the software.
• Conduct functional and physical configuration audits as needed.
• Participate in the change control process.
• Provide training and support on software configuration management process as needed on the use of version control.
• Assist in the development, analysis and maintenance of tools that support and automate processes for our software product releases.
• Assist engineers in the setup and configuration of their local development environments.
• Maintain control and account of all project assets.
• Design, document and establish configuration documentation.
• Perform routine software builds and resolve build conflicts and issues in a timely manner.
• Participate in software process improvement efforts.
• Verify the completeness and accuracy of release libraries and documentation before implementation.
• Develop and administer project baselines.
• Develop and maintain code versioning, branching and merging.
• Maintain multiple active branches and database versions at the same time while keeping source code and database schema synchronized.
• Responsible for the software tracking and releases.
• Manage the application and OS configuration files and settings for test environments.
• Provide deployable solutions and release instructions to operations staff for deployment and management.
• Other duties as assigned.
Qualifications:
Qualifications:
• Associates or B.S Electrical Engineering, Computer Science, Information Technology or equivalent experience required.
• 5-7 years of experience in Microsoft and Cisco environments including some experience with virtualization technologies.
• 3+ years in a Software Configuration Manager or Release Manager role required.
• 3+ years experience with Team Foundation Server or equivalent experience working with other version control/configuration management tools.
• Strong experience in Microsoft VisualStudio.NET development (Windows and Web applications).
• Experience or knowledge of continuous integration, application lifecycle management and automation and job scheduling systems.
• Experience with process automation using scripting languages.
• Thorough understanding of a project’s life cycle and development methods.
• Strong communication and interpersonal skills to interact with developers, QA teams, and functional managers
Using Business Intelligence to Drive your own Recovery.
June 29, 2010
eWeek published a video describing the value of using Business Intelligence to find and exploit market and revenue opportunities. Great point, and very well worth the 6:49 it takes to view it. Many organizations are using BI to understand some of the basic historical results of their business. It’s the next level of organization who begins to answer questions like the below using their BI toolset:
- What are my customer’s buying is a basic question, but moreover, what products do they buy together?
- Which products do they buy when times are tough?
- What did they buy during the last recovery?
- What aren’t they buying, and what should I recommend they buy?
All great questions, and clearly a value add of a strong BI platform.
eWeek – Using-Business-Intelligence-to-Find-Your-Economic-Recovery
Simplicity and Transparency
May 11, 2010
Last Thursday we had a great turnout for our first ever BI Symposium. Our host was the NKU METS Center. If you never been, you should check it out! It’s a wonderful facility! It’s truly state-of-the-art!
Our first speaker of the day was Dr. David Holcomb. David really set the tone well with his presentation on Simplicity and Transparency. Do you think of data as an asset? If so, treat it like the rest of your assets:
- Acquire it
- Prepare it
- Deploy it
- Manage it
So many times, we skip (or underfund) the “manage” step or at worse, skip all 4 steps and keep the data hidden from the organization.
David’s presentation can be found below. Enjoy!
– Jodie
BI and Software-As-A-Service (BI SaaS)
May 3, 2010
Who moved my cheese…again???
Economy challenges always seem to prompt new business models and productivity increases. Remember 10 years ago and the dot.com bomb?? Prior to 1999, websites were being developed in great numbers but there was no revenue model to support it. Those companies failed…others, that found a way to take a seemingly free service and get paid for it thrived. Additionally, with the fall off in the economy, people had to find a way to deliver the same services their customers were used to but do it for less. Voila! Off-shore resources!!
In the last several years though, even off-shore resources are expensive. Seasoned IT professionals (baby boomers) are retiring and taking valuable company info along with them. Profit margins for most companies continue to erode as spending has slowed. DASD has gotten significantly less expensive and bandwidth has quadrupled (or more?)! Those “free” websites now charge fees, but they aren’t outrageous. Given these changes, it makes sense that more and more applications are moving into the Cloud.
As you know, here at LUCRUM, “we do BI”. Respoinding to our customers, we implemented Agile BI concepts long before it was fashionable. We are able to get BI projects up and running in significantly less time than our “big 6″ competitors (and do it for less!). As we continue to investigate ways to get data to our customers faster, we have become fascinated with the Cloud. Certainly there has to be a way to take all of these company assets, secure them in the Cloud and give users better/faster access to their data.
We’ve investigated a few companies that are doing this today: Good Data, OCO, BIRST, and PivotLink. What’s interesting about each of these companies is that they’ve taken the common business problems – Sales and Finance – and created models to support them. I was fortunate to participate in a meeting with Good Data last week. I’m excited to learn more about each of these companies and even more excited to see how LUCRUM can support BI in the Cloud!
Stay tuned!
- Jodie
Business Intelligence Symposium May 6th!
April 6, 2010
Join us on Thursday, May 6, 2010 at the NKU METS Center for a half-day symposium of collaborative learning, focused on business intelligence. The Business Intelligence Symposium brings together regional business & IT executives to learn how their peers have been implementing data analytics, business intelligence solutions and Dashboarding. The emphasis of the symposium is to share ideas, stories, experiences, and business cards. Case studies, along with live demonstrations will be presented. Breakfast and lunch will be provided in a collaborative environment that facilitates peer networking and BI discussions for an enhanced learning experience.
View the agenda below and register today for $49 at the following link: http://tinyurl.com/yef3khh
Agenda:
7:30am – 8:00am Registration and Breakfast
8:00am – 9:00am David Holcomb, PhD – Director, Data Management, Western Union Simplicity and Transparency – How to do Effective Data Warehousing and Business Intelligence (Presentation)
9:00am -9:45am Mr. Steve Hangen – CIO, WinWholesale BI Roadmap – A Project, a Journey, a Culture (Presentation and Demo)
9:45am -10:00am Coffee Break & Conversations
10:00am – 10:45am Mr. John R. Ward – Director, Health Systems Integration, TriHealth The New Era of Healthcare Clinical Information Systems Unstructured Data – Internal/External
10:45am –11:30am Mr. Jeff Shaffer – Vice President of Legal Operations, Unifund Visualization – Running a business with Dashboards and Scorecards (Presentation and live Demo)
11:30am – 1:00pm Lunch /Panel Discussion led by Dr. David Holcomb and guest speakers
Business Intelligence Symposium Save the date!
April 2, 2010
LUCRUM Inc. is proud to announce The Business Intelligence Symposium being held on May 6th 2010 at the Northern Kentucky University METS Center http://www.themetscenter.com/center/default.aspx located near the Cincinnati/Northern Kentucky Airport. Check back for more details over the coming weeks.
Trusting Your Gut
March 29, 2010
In a recent McKinsey Article, “Strategic decisions: When can you trust your gut?”, Nobel laureate Daniel Kahneman and psychologist Gary Klein debate the power and perils of intuition for senior executives. Being in the business of business intelligence, I was at first shocked at the premise that there would be a time to trust your gut. After all, I’ve been telling execs for years that data, not intuition, should be the premise for making key business decisions. *Breathe* They agree.
What I found most interesting was that while they gave credibility to the strength of one’s intuition, they cautioned not to be seduced by overconfidence. They noted that most executives are promoted because of the confidence that they project which makes them more likely to be leaders in the first place. This confidence can cause others to trust the intuition of an exec because the exec believes they they themself are right. Hmmm…interesting point.
I think in my next BI engagement I’ll be adding a new dimension to the cube: Exec Intuition. We can plot that as a 1 (yes) or a 2 (no) and then analyze how often their intution is correct.
– Jodie
Polls
Loading ...Recent Comments
Tags
What we're talking about...
- RT @marksmithvr Blog: Kalido Does Data Governance Right http://bit.ly/b2mQmR #DataGovernance #Kalido #CIO #MDM #Data #DataQuality 2010-06-22
- Figured out folders. Love that! #iPhone OS4 2010-06-22
- Upgraded to iPhone OS 4. When will I see multitasking? 2010-06-22
- Shocking?!? RT @OneMicrosoftWay Microsoft puts Messenger on the iPhone's App Store - http://arst.ch/lcy 2010-06-22
- Been off of Twitter lately due to lack of access at new client. Boy have I missed a lot! 2010-06-22
- More updates...