Prior to joining Birst, I spent the several years at my previous position evaluating and implementing multiple analytics projects. Throughout the process, I learned valuable lessons that I think are helpful to share with others looking for analytics solutions and are in need of a BI vendor. In Part 1 of this blog series, I shared tips on how to start the process of finding qualified BI vendors that fit your needs and walked through my own evaluation methods to help you identify which vendors you’ll want to take to the next step in the process: price comparison and proof of concept trials.
The Challenge of Vendor Price Comparison
One consideration I didn’t anticipate when I began my very first BI implementation process was how different pricing models would vary from vendor to vendor. There may be others now, but the models I encountered included:
- Pay per user;
- Pay per server/CPU;
- Pay based on volume of data;
- Pay a percentage of your product’s base cost to the BI vendor; and
- Pay a straight annual fee and pay for services.
The difference in the pricing models makes it difficult to compare costs across systems. In my case, I solved the problem by calculating the cost of implementation through the end of year one, and then the cost of each year past year one. It was critical to establish the project assumptions based on the goals and personas. My assumptions on our first project included:
- 20 million rows of data (we could control this number to limit our potential cost);
- 1 refresh of the data per day;
- Professional services to help determine what data we needed, build the loading processes, create the initial charts, and teach us how to use the system; and
- Two changes to the underlying data model per year.
Using this information, I approached each of the seven BI vendors on my short-list and asked them to calculate the costs for the first five years. If you use this approach — asking the sales team do the costing for you — you’ll save a lot of time.
With all the permutations for pricing that exist, I can’t overstate how vital it is that your selected vendor’s pricing model matches your intended product-offering model. If it’s incorrect, you can limit the potential ways in which you can offer the product to your users. As an example, we wanted to include basic BI for all existing customer accounts, all users, but with limited data. If we had chosen a vendor that based pricing on the number of users, we’d be in the position of either reducing margin as more people used the system or trying to limit the total users per account. This was the opposite of what we wanted to achieve — we wanted everyone using the system and getting hooked!
As a result, for our project, the vendors that based pricing on user counts were eliminated. Further, our product also had a very thin profit margin, so we eliminated the vendor who wanted a percentage of our revenue from further consideration.
Moving on to Proof of Concept Trials
Following the evaluation method I’ve outlined, at this point, you should have a good idea of projected vendor costs and your list of vendors left to compare will be short. For my initial project, I had narrowed my list down to three vendors. Now it is time to see the applications in action using your data and your desired charts.
The Proof of Concept (POC) trials are designed to mimic what you might see in a production situation — similar data, similar charts — just less data and no integration. The goal is see how quickly each vendor can can get up and running with your data, as well as how quickly they would respond to changes to the requirements and the little challenges that always pop up in a complex project. The vendor that we eventually chose was able to perform not one, but three separate trials for our team faster and with less stress than the second place vendor was able to get anything up and running.
As an added bonus, the POC allows you to “test” the personalities of the implementation team. You will be working with their team for the next 90 days or so — this is a great opportunity for a preview. For example, if problems arise over the weekend when the demo for the CEO is on Monday morning, would they be willing and able to help? Do they quickly pick-up on your business challenges, or do you need to explain your company’s business model repeatedly? In my opinion, the answers to these questions are as valuable as the technical aspects of the POC and should factor into your decision. In my case, we ended up choosing the vendor that had great capabilities, a great team, moved quickly, and felt more like a partner than a vendor.
Following the POC, you will likely have a clear idea of which BI vendor you will choose. In Part 3 of this blog series, I’ll provide insight into vendor contracts and, finally, project implementation. Thanks for reading, and stay tuned!