Several years ago, I had just accepted a new role at a small technology company when an email from my soon-to-be boss changed the direction of my career:

“We’ve got all this data we’ve gathered from our application. We should do something with it, like adding some sort of business intelligence. Start thinking about what we could do.”

At the time, I had no idea just how little I knew about selecting a business intelligence vendor, getting the system implemented and integrated, and creating an actual product offering based on BI capabilities.

After three years and the creation of six embedded analytics products, I’ve learned some valuable lessons. For me that first BI project was an exciting experience — I had no concept of what was around each corner and how each decision would affect the process. Since then, I’ve implemented business intelligence into existing products multiple times and have learned more with each additional project.

Now that I work at Birst, I thought I’d to share the mistakes, lessons learned, and best practices gained through trial and error that ultimately helped make my implementations successful. You will likely make plenty of missteps with your own project, but hopefully you can avoid the pitfalls I encountered by following the techniques and recommendations in this blog series. The first in the series, Part 1, focuses on identifying a list of vendors to begin the evaluation process.

Where to Begin?

A quick search on Google for “business intelligence tools” returns 110 million search results. Assuming 10 percent of those results are for BI tools, and that you review two vendors per day, and that you have a team of five people to help in the assessment process — it would take you 30,000 years to complete your selection process. This is a problem.

How do you cull through the massive number of BI offerings available today in a reasonable amount of time without missing key players? My recommendation is to start with the experts: Industry analyst reports. These professionals have already done the hard work for you, conducting extensive background, evaluating the major players and talking in-depth to both the vendors and their customers. In my opinion, the best place to start, and where I ultimately began my own research, is with the Gartner Magic Quadrant for Business Intelligence and Analytics. (You can download a complimentary report from Birst here)

The report includes a comprehensive listing of all the vendors you should be reviewing — a master list from which to start narrowing down solution candidates. But how will you know who to include and exclude from your search? Even the Gartner report has too many options for you to fully demo each application. To narrow down the list of vendors, start by defining your goals and personas: Who will be using the business intelligence platform and for what purpose?

One of the initial mistakes I made was that I didn’t frame our goals crisply. We had started early on by listing project goals, so we knew:

  • We wanted to offer BI as part of our core product;
  • It had to be embedded in an existing SaaS application and maintain the look and feel of our application;
  • It had to be easy to install and maintain (at the time, we didn’t have many Engineering resources available to help in the implementation);
  • We had to be able to control costs as the application grew; and
  • It had to have a great user experience — both for the user creating dashboards and for the end user consuming the analytics.

At the time, I thought I had a good handle on exactly what we needed. While I knew a lot, I didn’t realize until later that I didn’t know enough. Basic project goals are necessary, but not totally sufficient for BI implementation.

For the second analytics implementation project, I corrected a major oversight from the first: I created personas describing exactly who would be using the system. By persona, I don’t mean a marketing persona: “35-45 year old male who lives in San Diego and an enjoys the soulful sounds of the Backstreet Boys.” Rather, I mean a user experience persona that describes the user you are serving and the business problems they need to address.

It is important to fully define these personas because your BI application will likely have a wide-range of users — from tactical users to executive-level strategic users. I overlooked this step during the first project and the result was a co-mingling of tactical and strategic charts and graphs on the same dashboards. I ended up creating pages that didn’t quite fit any specific user type well. For example, charts that made sense for a front-line manager were useless to the COO.

Therefore, for my second implementation project, I defined the personas that would use our system as:

  • Chief Marketing Officer
  • Sales Manager
  • Head of Customer Advocacy

For each of these personas, we created a quick persona brief to get us thinking about the users that needed to be served by each dashboard we would create. I recommend you start your process of vendor selection by making a list of the project goals and then think about the specific users who will benefit from the use of analytics.

The Process of Elimination: Identify Your Top Three Vendors

At this point in the process you will have three key pieces of information: A list of vendors from which to choose; a list of your project goals; and a list of personas that you need to serve.

The next step is to eliminate those vendors whose products don’t align with the needs of your users. For example, at my company we quickly eliminated several “Traditional BI” vendors because although they had well-established products, they couldn’t operate embedded into our web-based application. Then, from the twenty initial vendors on our initial list (selected via the Gartner report), we narrowed it down to seven potential solutions by eliminating those that couldn’t fit with our operating model. We continued the elimination process by evaluating the remaining candidates against a combination of factors.

To simplify this process, I recommend creating a matrix of both objective and subjective criteria. List the project goals as criteria, but also add items such as “ease of working with the vendor”. Do not overlook the subjective criteria. While meeting the basic technical requirements is critical, it’s the subjective criteria that will greatly impact your life over the next few months.

As an example, during our initial discussion with a particular vendor, the salesperson told our head of engineering that he should go back and read the product specs web page so that he’d be able to ask better questions on the next call. That incident gave us a hint into what we might expect working with the vendor’s team and factored heavily into our “ease of doing business” category. And there wasn’t another call.

Further, for each vendor candidate, I suggest preparing a one-pager that includes criteria such as:

  • Years in business;
  • Size of the vendor;
  • Number of implementations to date;
  • Number of embedded or “Powered by” type implementations;
  • Cost of year 1 (setup plus service fees);
  • Cost years 2-5 (mostly the on-going service fees);
  • Cost of professional services for implementation; and
  • Support ecosystem (did they provide training, marketing support, etc.).

After combining all of these factors into your evaluation, review the list and identify the top three business intelligence vendors that you feel best fit your needs and want to move forward with in the process. The next blog in this series, Part 2, will help you determine price comparison and walk you through the next step in vendor evaluation: Proof-of-concept trials.