Fundamental Pathway to Business Intelligence and Analytics: Strategy, Steps, Processes, and Tools
All businesses operate with data — information that is generated from many of your company’s internal and external sources. These data channels act as a pair of eyes for executives, providing them with analytical information about what’s going on in the business and the market. Accordingly, any misunderstanding, inaccuracy or lack of information may lead to a distorted view of the market situation as well as of internal processes — followed by bad decisions.
Making data-driven decisions requires a 360° view of all the aspects of your business, even those you didn’t think of. But how to turn unstructured data chunks into something useful? The answer is business intelligence.
In this article, you will understand the actual steps of bringing business intelligence into your existing corporate infrastructure. You will learn how to set up a business intelligence strategy and integrate tools into your company workflow.
What is business intelligence?
Let’s begin with a definition: business intelligence or BI is a set of practices of collecting, structuring, analyzing, and turning raw data into actionable business insights. BI considers methods and tools that transform unstructured data sets, compiling them into easy-to-grasp reports or information dashboards. The main purpose of BI is to provide actionable business insights and support data-driven decision making.
The whole process of business intelligence can be divided into four stages:
- Data gathering
- Data cleaning/standardization
- Analysis
- Reporting
The biggest part of the BI implementation is the use of actual tools that perform data processing. Different tools and technologies form a business intelligence infrastructure. Most often, the infrastructure includes the following technologies that cover data storing, processing, and reporting:
- Data sources
- ETL (Extract, Transform, Load) or data integration tools
- Data warehouse
- Online analytical processing cubes
- Data marts
- Reporting (BI) tools
Business intelligence is a technology driven-process that relies heavily on the input. Technologies used in BI to transform unstructured or semi-structured data can also be used for data mining, as well as being front-end tools to work with big data.
Business intelligence and predictive analytics
The definition of business intelligence is often confusing as it intersects with other spheres of knowledge, especially predictive analytics. One of the biggest errors is using business intelligence and predictive analytics terms interchangeably.
Basically, business intelligence is a data analytical approach, answering the questions what was happening? and what is happening?. This type of data processing is also called descriptive analytics. With the help of descriptive analytics, businesses can study the market conditions of their industry, as well as their internal processes. Historical data overview helps to find a business’s pain-points and opportunities.
Predictive analytics is concerned with forecasting based on data processing of past events. Instead of producing overviews of historical events, predictive analysis makes forecasts about future business trends. Those predictions are based on past events analysis. So, both BI and predictive analysis can use the same techniques to process data. To some extent, predictive analytics can be considered the next stage of business intelligence.
Both analytical approaches refer to three main types of data management:
- Descriptive analytics (BI)
- Predictive analytics
- Prescriptive analytics
Prescriptive analytics is the third type that aims at finding solutions to business problems and suggests the actions to solve them. Currently, prescriptive analytics is available via advanced BI tools, but the whole area hasn’t developed to a reliable level yet.
So here is the point, when we start talking about the actual integration of BI tools into your organization. The whole process can be broken down into the introduction of business intelligence as a concept for your company employees and the actual integration of tools and applications. In the next sections, we will walk through the key points of BI integration into your company and cover some pitfalls.
Step 1: Introduce business intelligence to your employees and stakeholders
Let’s begin with the basics. To start utilizing business intelligence in your organization, first and foremost explain the meaning of BI with all your stakeholders. Depending on the size of your organization, frames of the term may vary. Mutual understanding is vital here because employees of various departments will be involved in data processing. So, make sure that everybody is on the same page and doesn’t confuse business intelligence with predictive analysis.
Another purpose of this phase is to pitch the concept of BI to the key people that will be involved in data management. You will have to define the actual problem you want to work on, set KPIs, and organize required specialists to launch your business intelligence initiative.
It’s important to mention that at this stage, you, technically, will make assumptions about the sources of data and standards set to control the data flow. You’ll be able to verify your assumptions and specify your data workflow at the later stages. That’s why you must be ready to change your data sourcing channels and your team lineup.
Set the objectives, KPIs, and requirements
The first big step after aligning the vision would be to define what problem or group of problems you are going to solve with the help of business intelligence. Setting the objectives will help you determine further high-level parameters for BI such as:
- What sources of data will be used? (CRM, ERP, website analytics, external sources, etc.)
- What type of data do we need to source? (sales numbers, reports, website traffic, etc.)
- Who needs access to this data? (top management, market analysts, other roles)
- What types of reports do we need and how must they be presented? (spreadsheets, diagrams, ad hoc reports, or interactive dashboards)
- How would progress be measured?
Along with the objectives, at that stage, you will have to think of possible KPIs and evaluation metrics to see how the task is accomplished. Those can be financial restraints (budget applied to development) or performance indicators like querying speed or report error rate.
By the end of this stage, you must be able to configure the initial requirements of the future product. This can be a list of features in a product backlog consisting of user-stories, or a more simplified version of this requirement document. The main point here is that, based on the requirements, you should be able to understand what architecture type, features, and capabilities you want from your BI software/hardware.
Step 2: Choose tools or consider a custom solution
Compiling a requirement document for your business intelligence system is a key point to understand what tool you need. For large businesses, building its own custom BI ecosystem can be considered for several reasons:
- Enterprise level organizations may not entrust their valuable data to a third party.
- BI tools mostly differentiate by serving the needs of some specific industry. There might be no vendor on the market that provides services for your industry.
- And the last reason is that processing large volumes of information or working with big data may serve a good reason to initiate custom BI development, instead of looking for the vendor, since you may have higher flexibility in terms of choosing a cloud infrastructure provider.
For smaller companies, the BI market offers a great number of tools that are available both as embedded versions and cloud-based (Software-as-a-Service) technologies. It’s possible to find offers that cover nearly any kind of industry-specific data analysis with flexible possibilities.
Based on the requirements, your industry type, the size, and needs of your business, you will be able to understand whether you are ready to invest in a custom BI tool. Otherwise, you can choose a vendor that will carry the implementation and integration burden for you.
Step 3: Gather a business intelligence team
The next step would be gathering a group of people from different departments of your company to work on your business intelligence strategy. Why would you even need to create such a group? The answer is simple. BI team helps to gather representatives from different departments to simplify communication and get department-specific insights about required data and its sources. So, the lineup of your BI team should include two main categories of people:
Domain representatives from different departments
These people will be responsible for providing the team with access to data sources. They’ll also contribute their domain knowledge to choosing and interpreting different data types. For instance, a marketing specialist can define whether your website traffic, bounce rate, or newsletter subscription numbers are valuable data types. While your sales representative can provide insights into meaningful interactions with customers. On top of that, you will be able to access marketing or sales information via a single person.
BI-specific roles
The second category of people you want on your team are BI-specific members that will lead the development process and make architectural, technical, and strategic decisions. So, as a required standard you will need to determine the following roles:
Head of BI. This person must be armed with theoretical, practical, and technical knowledge to support the implementation of your strategy and actual tools. This can be an executive with knowledge of business intelligence and access to data sources. The Head of BI is a person that will make decisions to drive implementation.
BI engineer is a technical member of your team who specializes in building, implementing, and setting BI systems. Usually, BI engineers have software development and database configuration background. They also must be well-versed in data integration methods and techniques. A BI engineer may lead your IT department in implementing your BI toolset.
The data analyst should also become a part of the BI team to provide the team with expertise in data validation, processing, and data visualization.
Step 4: Document your BI strategy
Once you have a team and you’ve considered the data sources required for your specific problem, you can start developing a BI strategy. You can document your strategy using traditional strategic documents such as a product roadmap. Business intelligence strategy may include various components depending on your industry, company size, competition, and business model. However, the recommended components are:
Data sources
This is documentation of your chosen data source channels. These should include any types of channels, whether it is a stakeholder, analytics of the industry in general, or the information from your employees and departments. Examples of such channels may be Google Analytics, CRM, ERP, etc.
Industry/custom KPIs
Documenting standard KPIs of your industry as well as your specific ones may open the fullest picture of your business growth and losses. Ultimately, BI tools are created to track these KPIs supporting them with additional data.
Reporting standards
At this stage, define what kind of reporting you require to extract valuable information conveniently. In case of a custom BI system, you may consider visual or textual representations. If you have already chosen the vendor, you may be limited in terms of reporting standards, as vendors set their own. This section may also include data types you want to deal with.
Reporting flow type and end-users
An end user is a person who will observe data through the interface of the reporting tool. Depending on the end users, you may also consider a reporting type flow:
Traditional BI. Traditionally, BI was designed for executives only. Since the number of users and types of data is limited, there’s no need for full automation. So, a traditional BI flow type requires technical staff as an intermediary between the reporting tool and the end user. If an end user wants to extract some data, he or she has to make a request and tech staff will generate a report from the required data. In this case, your IT department acts as a power user, a user that can access data and influence its transformation.
The traditional approach offers a more secure and controlled data flow. But, relying on the IT department may introduce a lag in flexibility and speed in case of processing big amounts of data (especially for big data). If you strive for more report control and precision of reports, form a dedicated IT team to take care of queries and report formation.
Self-service BI. Today, modern companies and solution providers utilize self-service BI. This approach allows business users as well as executives to get the reports that are automatically generated by the system. Automated reporting doesn’t need power users (admins) from your IT to process each request to your data warehouse; however, technical staff is still required to set up the system.
Automation may lower the quality of the end reports and their flexibility as it will be limited by the way the reporting is designed. But, as a benefit, the self-service approach doesn’t require actual technical staff to operate in the system all the time. Users that are not tech-savvy will be able to serve a report for themselves or access a dedicated section of the data storage.
Step 5: Set up data integration tools
The integration phase of the actual tools will require a lot of time and work by your IT department. There are various structural elements of a BI architecture you will have to develop in case you want to create a custom solution for your business. In other cases, you are always free to choose a vendor from the market that would carry implementation and data structuring for you.
One of the core elements of any BI architecture is a data warehouse. The warehouse is a database that keeps your information in a predefined format, usually structured, classified, and purged of errors. If your data isn’t preprocessed, your BI tool or your IT department won’t be able to query it. For this reason, you can’t directly connect your data warehouse with your sources of information. Instead, you must use ETL (Extract, Transform, Load) tools or data integration tools. They will preprocess raw data from the initial sources and send it to a warehouse in three consecutive steps:
- Data extraction. The ETL tool retrieves data from the data sources including ERP, CRM, analytics, and spreadsheets.
- Data transformation. Once extracted, the ETL tool starts data processing. All extracted data is analyzed, have duplicates removed, and then is standardized, sorted, filtered, and verified.
- Data loading. At this phase, transformed data is uploaded into the warehouse.
Usually, ETL tools are provided out of the box with BI tools from vendors. (The most popular one is covered below).
Step 6: Configure a data warehouse and choose an architectural approach
Data Warehouse
Once you’ve configured data transmission from the chosen sources, now you have to set up a warehouse. In business intelligence, data warehouses are specific types of databases that usually store historical information in SQL formats. Warehouses are connected with data sources and ETL systems on one end and reporting tools or dashboard interfaces on the other. This allows for presenting data from various systems via a single interface.
But a warehouse usually contains extensive amounts of information (100GB+), which makes it reasonably slow to respond to queries. In some cases, data can be stored unstructured or semi-structured, which leads to a high error rate when parsing data to generate a report. Analytics might require a certain type of data grouped in one storage space for ease of usage. That is why businesses use additional technologies to provide faster access to smaller, more thematic chunks of information.
There are various types of solutions used to present analysts with smaller portions of a warehouse. The most used of them are Online Analytical Processing and Data Marts. These technologies offer quicker reporting and easy access to the required data.
Recommendation: if you don’t have large volumes of data, the utilization of a simple SQL warehouse would be enough. Additional structural elements like data marts will cost you a lot without providing any value. This option fits small businesses or industries that operate relatively small amounts of data.
Data Warehouse + Online Analytical Processing Cubes
Data stored in a warehouse has two dimensions, as it’s usually depicted in a spreadsheet format (tables and rows). So, the way a warehouse stores data is also called a relational database. It may include thousands of data types in one database, so querying a data warehouse takes a significant amount of time. To satisfy analyst needs to access data quickly, analyze it from different dimensions, and group whenever they need it, OLAP cubes are used.
OLAP or online analytical processing is a technology that processes data and gives access to it from multiple dimensions at a time. Structuring your data in cubes helps to overcome the limitations of a data warehouse.
OLAP cube is a data structure optimized for quick data analysis from SQL databases (warehouse). Cubes source data from a data warehouse being a smaller representation of it. However, the structure of data assumes that there are more than 2 dimensions (row and column format of spreadsheets). Dimensions are the crucial elements that form the report, e.g. for sales department it might be:
- Salesperson
- Sales amount
- Product
- Region
- Time period
Cubes form a multidimensional database of information that can be adapted to group it in different ways and create reports more quickly. OLAP cubes dedicated to different data themes form OLAP databases. A warehouse and OLAP are used in conjunction, as cubes store a relatively small amount of data and serve for processing convenience.
Recommendation: The data warehouse + OLAP cubes architecture can be considered a typical one. It can be used by companies of all sizes that require data storage and complex multidimensional analysis of the information. If you don’t want to bombard your warehouse with queries, consider an OLAP architectural approach.
Data Warehouse + Data mart technologies
A warehouse is the first and biggest element of business intelligence architecture. A smaller representation of warehouses’ datasets is a data mart. A data mart is a subject-oriented piece of a warehouse that gathers thematically familiar information dedicated to a specific department. With the help of data marts, separate departments can access required data, as data marts offer insights dedicated to a single sphere of business. That means your developers can avoid setting up permission-based querying for the end users.
Recommendation: Data warehouse + Data marts is the second most popular architectural style based on the utilization of data marts to distribute the required information to each department. This approach can be used to establish constant reporting or easy access to information, without providing permissions to end users.
Hybrid architecture
Enterprise businesses may require multiple options for data management. Data marts and cubes are different technologies, but they are both used to represent smaller chunks of information from the warehouse. Data marts represent a problem-specific subset of a data warehouse, but it can be implemented differently. The implementation option includes relational databases (warehouse or any other SQL database), and multidimensional, which are basically OLAP cubes. So, you can use both technologies to manage your data and distribute across the organization’s departments.
Recommendation: You can utilize both technologies as they support the same idea, but serve different purposes. Data marts can be implemented as a part of a data warehouse for security, data aggregation, or accessibility. Or you can use data marts as a representation of several dimensions of the OLAP cube. But, keep in mind that both data marts and OLAP cubes will require separate database setup.
Step 7: Implement the end-user interface: reporting tools and dashboards
Formed into digestible, thematically related chunks of information in Online Analytical Processing cubes or data marts, the data is finally presented via a user interface of BI tools. That is where descriptive analysis brings its value to the end user.
Modern BI tools offer several ways to present the required data. In the past, business intelligence could produce only static reports, based on future and past events. Today, BI is capable of producing interactive dashboards with customizable portions of information. But templated reporting remains the most popular method of data presentation.
The most valuable way of presenting information is considered ad hoc reporting. Ad hoc reporting allows users to go deeper into a standard report by leveraging any kind of data for single usage. This type of reporting is used instead for daily or weekly reports, as a fuller version, because a user would pull data from a warehouse (cube or data mart) right at the moment of viewing the report. That guarantees the freshest information presented by querying databases for every single piece of information. So, basically, ad hoc reporting is a customizable real-time report used to find the answer to one specific business question.
Step 8: Conduct training for end users
To make the onboarding process smooth for your employees, we strongly recommend conducting training sessions. Those sessions may have a different form: if you use an embedded analytical tool in your CRM or ERP, you can use onboarding practices like video-hints or interactive onboarding tools that lead users through steps.
If you don’t have a budget to automate training, you still have to take care of training provided by a manager or members of your BI team.
Key business intelligence tools on the market
It’s important to mention that BI tool providers usually supply users with data integration, ETL, reporting tools (dashboards), as well as warehousing services. It means that, most often, you would get the whole BI architecture integrated into your system. So, here are some examples of business intelligence tools providers.
Sisense
Sisense is one of the biggest names in the business intelligence market. Their product offers back-end and front-end access to data-analysis systems for users of different technical levels. Sisense also offers data-storage services, making it a one-stop solution. The pricing model is a yearly subscription, but the fees may vary depending on the numbers of users, amount of data, and type of project.
Zoho Analytics
Another big name in the business intelligence industry is Zoho Analytics. Zoho offers a full infrastructure for both small and large businesses with a scalable interface. Among the useful features, it offers are open RESTFUL APIs to connect all your required CRS and ERP systems, collaborative workspaces to share with your employees or stakeholders.
Tableau
Tableau is a cloud-based BI solution that pioneered drag-and-drop interfaces for reporting tools. Tableau software also has collaborative features: a single sign-in page can be created for your analysts to access the dashboard and share the information. You can query the data to be sent to your mobile device. A dedicated Tableau application can modify reports and save changes right from your phone.
SAP
SAP is an international company that offers many technical solutions, including SAP Business Objects Business Intelligence suite and Cloud Analytics products. The first product is a basic solution for all business sizes. The platform offers smart querying and ad hoc reporting. Additionally, available dashboard reporting works in a role-based format, meaning any user can set up an analytical dashboard depending on the user’s role. The cherry on top is SAP products can be easily integrated with Microsoft Office products.
BusinessQ
The last but not the least, BusinessQ with its BI solutions tailored specifically for small- and medium-sized business. BusinessQ offers both a standalone web application as well as an embedded version to be built into your own application.
Domo
Domo BI platform is a cloud-first solution, aimed at businesses of all sizes. The service is scalable, allowing it to work with big data or small corporate databases. Domo offers access to real-time dashboards, using data marts implemented on OLAP cubes to allow multidimensional analysis and data dedication by departments.
Qlik
Qlik is a business intelligence provider that offers various products for data visualization, interactive dashboarding, and self-service reporting. The infrastructure can be implemented in a cloud or on-premise. Additionally, Qlik offers access to a list of public data sets to source information from.
Summary
Business intelligence tools have been around for more than 20 years. However, the look and basic functionality of a “standard” BI tool have changed a lot. Instead of just static reporting, now each vendor offers ad hoc reporting or interactive dashboards for analysts to collaborate on. Additionally, self-service BI becomes a standard for average business tasks, allowing entrepreneurs to conduct analytics with help of modern data warehouse techniques, more cost-effectively. Following general tech trends, the most recent features to be introduced in BI are cloud-based platforms and mobile BI reporting.
So, knowing the general trends and technologies used in the industry, you will be able to build your own custom BI systems or choose an existing one to provide easy-to-grasp reports that support your decisions. Business intelligence is no longer a privilege for an executive; today it’s a collaborative tool for your whole organization. Make sure you pick the right vendor and include all the necessary features to help your employees access those insights.
