OVERVIEW OF BI & ANALYTICS

OVERVIEW OF BI & ANALYTICS.

[Audio] It is no surprise that Business intelligence and business analytics are two of the fastest growing market domains in the world. Organization are generating data at a rapid rate. There is a need to use this business data and make smarter decisions, companies are looking for methods and tools to turn business data into actionable insights. This is where business intelligence and analytics plays a critical role. Therefore, it becomes very important to understand and learn about these two domains..

[Audio] Hi Everyone I welcome you to this session on Business Intelligence and analytics course which contain everything you need to know. Now before we move any further let's take a look at the agenda. The first module which is introduction to Business Intelligence and business analytics will help you to understand what is BI and analytics, its significance and why it is useful. Along with this we will also cover Data, Data Analysis, Data modelling and dig down to the grass root level. We will also explore Entity relationship diagram where we will go through entities, attributes ad various relations. At the end we will also learn about data integration including ETL and ELT. This will help you to get familiar with various concepts and fundamentals related to this realm. So, lets begin….

[Audio] Business Intelligence Business Intelligence is a group of tools, methods, and programs that are designed to collect, analyze and transform data into valuable information ( insights) that companies can use to make better decisions and optimize their businesses It is a suite of software and services to transform data into actionable intelligence and knowledge. BI has a direct impact on organization's strategic, tactical and operational business decisions. BI supports fact-based decision making using historical data rather than assumptions and gut feeling. Business intelligence uncovers insights for making strategic decisions. Business intelligence tools analyze historical and current data and present findings in intuitive visual formats. Now we will understand what is analytics.

[Audio] Analytics Analytics is the process of discovering, interpreting, and communicating significant patterns in data. Simply, analytics helps us see insights and meaningful data that we might not otherwise detect. Basically, analytics is the science of analyzing raw data to make conclusions about that information. Analytics provides us with meaningful information which may otherwise be hidden from us within large quantities of data. It is something that any leader, manager or just about anyone can make use of especially in today's data-driven word. Information has long been considered as a great weapon, and analytics is the forge that creates it. Analytics changes everything, not just in the world of business, but also in science, sports, health care and just about any field where vast amounts of data are collected..

[Audio] Although business intelligence is utilized in different ways and for different purposes by individual companies, the process is uniform throughout all industries and typically unfolds as follows: Data from various sources – including internal company data and external market data – is collected, integrated, and then stored. Because "big data" is generally used, data is commonly stored in a data warehouse, created by a data engineer. Data sets are created and prepared for data analysis, often by creating data analysis models. Data analysts run queries against the data sets or models. The results of queries are used to produce visualizations in the form of charts, graphs, histograms, or other visual representations, along with BI dashboards and reports. Decision-makers utilize the data visualizations and reports to help them in making decisions; they may also use their BI dashboard to probe further into the data for more information..

[Audio] Although Business intelligence is all about how precisely you can use it for excavating values and benefits but superficially, we can consider the following benefits for reference. Implementation in any department or industry A business executive or manager might need access to data across the company, including marketing and sales, that will help him/her to make a better, more informed decision. The possibility that a business intelligence solution has on offer focuses on implementation in any industry, any function, and any data source your company might have or need. That means that a comprehensive overview can be obtained by simply implementing one of the best BI tools for your concrete business case. Includes artificial intelligence and predictive analytics As we all know it, managing large volumes of data on a daily basis has become a big challenge. The goal of a smart BI solution is to automate and quickly perform analysis of enormous datasets that will empower users to focus on what truly matters: actionable insights. That means that traditional means of collecting and analyzing data such as spreadsheets no longer bring added value, but challenge organizations in their time management and costs allocation, often much higher than planned. There is simply too much complex data to analyze and, consequently, companies must turn to modern business intelligence system solutions that have AI and predictive analytics on offer. Designed for both data analysts and average business users Modern self-service BI tools provide options tailored for average business users, data analysts, or advanced data scientists. They need to capture different business modes that will enable each employee, no matter the background, to be able to fully comprehend and create a data story, oftentimes visualized in a striking online dashboard. Provides real-time monitoring and access to data, anywhere, anytime Turning to BI means many standard, tedious processes such as exporting numerous spreadsheets can be automated through creating powerful dashboards accompanied by a number of interactive charts that you can adjust and incorporate with just a few clicks. These dashboards provide access to real-time data, refresh automatically, and have interactivity in nature to enable users to simply click on a part of the dashboard, and explore the data further..

[Audio] With the help of Business Intelligence and analytics organizations these days dig down into their data. Through that analyzed data they are able to find any event which has happened in the past and along with this how that event is affecting the present. These days because of continuous Business intelligence advancements, reasons behind occurrence of any past or present event can be identified easily. Modern Business intelligence and analytics tools can predict and forecast upcoming events or uncertainties, which may not be precise but accurate up to an extent..

[Audio] Although business intelligence and business analytics works in simulation with each other but their core is little different. The first difference between the two concepts is in the actual definition. While both of them serve a purpose in the analytical process, they don't serve the same one. A fundamental differentiation factor is in the method each of them uses as a base. While BI tells you what has happened in the past and what is happening now (descriptive analytics), BA tells you what will happen in the future ( predictive analytics). Usage is another factor that can help us understand how BI and BA differ from each other. In this case, not only the end-user changes but also the purpose of use. As we mentioned time and time again, BI tools are used with the purpose of reporting on the current and past performance of the organization, BA tools allow you to take it a step further by helping you decide on your next steps. Our third and last differentiating factor is in the application. Your data is used differently depending on whether you are conducting BI or BA analysis. While BI arranges the information in easy-to-understand reports, BA takes it a bit further than reporting..

[Audio] As we have understood business intelligence and analytics, now let's move on to Data first. Because without data, these processes not even exist. Here we will cover data and its types along with various types of data analysis..

[Audio] Data is raw piece of information that is capable of being moved and store. It can be in the form of numbers, words, pictures, digits etc. In the technical era data plays a significant role to compete in the market. Many multinational companies exist in global which are only storing data and providing it to others. By this only they are minting huge amount of money. We as an individual user generate ample amount of data at every moment. From searching something on google to clicking on any website link, sending messages, using social media and calling also, at every instant data is being generated and stored in multiple data bases. As an individual we are generating this much data, just imagine how much data is being generated in the world at every second. This is the power of data. It provides us with the facts. By organizing this data in different ways, we can make it easier to read, answer questions and solve problems..

[Audio] Basically, there are three types of data based on their format. Unstructured Data Semi structured data Structured data Lets discuss these in detail..

[Audio] Unstructured data is defined as data present in absolute raw form. This data is difficult to process due to its complex arrangement and formatting. Unstructured data management may take data from many forms, including social media posts, chats, satellite imagery, IoT sensor data, emails, and presentations. to organize it in a logical, predefined manner in a data storage. Unstructured data is a data which is not organized in a predefined manner or does not have a predefined data model; thus, it is not a good fit for a mainstream relational database. Example: Word, PDF, Text, Media logs. Next is semi structured data..

[Audio] Semi-structured data is a type of data that has some consistent and definite characteristics. It does not confine into a rigid structure such as that needed for relational databases. Semi-structured data is information that does not reside in a relational database but that has some organizational properties that make it easier to analyze. Example: XML data. Now lets look structured data which is widely used.

[Audio] Structured data is information that has been formatted and transformed into a well-defined data model. It concerns all data which can be stored in database in a table with rows and columns. They have relational keys and can easily be mapped into pre-designed fields. Today, those data are most processed in the development and simplest way to manage information. Example: Relational data. Now lets discuss what is data analysis.

[Audio] Data analysis is defined as a process of cleaning, transforming, and modelling data to discover useful information for business decision-making. The purpose of Data analysis is to extract useful information from data and taking the decision based upon the insights. As it is clearly visible in the data analysis process diagram, process has five important stages. First is identify the data which you want for analysis, or which is mandatory to overcome your business problem. Now, as you have identified the data second stage is to collect the data from various sources. It can be past company data, survey data etc. The collected data may have some error or not be in proper format. For eliminating these errors, it should be cleaned before analysis. Data cleaning is done through query languages. After data cleaning, the data can be analysed as per need to find insights. The analysis findings and insights need to be interpreted properly to solve the business problem. So whole data analysis process will not be of any worth if it is not interpreted properly..

[Audio] Here we will see important types of data analysis which is descriptive analysis, diagnostic analysis, predictive analysis and prescriptive analysis..

[Audio] The first type of data analysis is descriptive analysis. It is at the foundation of all data insight. It is the simplest and most common use of data in business today. Descriptive analysis answers the "what happened" or "what is happening" by summarizing past data, usually in the form of dashboards. Descriptive analytics gives you information on what is happening let us see a particular example this report gives us information about sales of ice cream X in different regions and it's also compared these sales for years 2015 and 16 this is a typical descriptive analytics support which gives you information on what is happening to sales of this ice cream so this chart one could infer that except for region 2 the sales have increased from 2015 to 16 so this is the level of information one could obtain from descriptive analytics. Now that we know the sales have dropped in region 2 our next job is to move to the next level of analytics which is diagnostic analytics. Lets move on to diagnostic analysis.

[Audio] Diagnostic analysis takes the insights found from descriptive analytics and drills down to find the causes of those outcomes. Organizations make use of this type of analytics as it creates more connections between data and identifies patterns of behavior. A critical aspect of diagnostic analysis is creating detailed information. This begins give you information on why is it happening. So our job here is to deep dive into region 2 and see what exactly is going wrong for us so this is a typical report for the market share of region 2 sometimes one could infer that for all brands X Y and is it the market share is going down because of the reason a new local brand has come up in 2016 which is gaining the market share this is the level of information you will be able to obtain using diagnostic analysis now that we know what is happening to our sales why is it happening our next step is to go to the next level of analysis which is called predictive analysis. Next move on to predictive analysis.

[Audio] Predictive analysis attempts to answer the question "what is likely to happen". This type of analytics utilizes previous data to make predictions about future outcomes. This type of analysis is another step up from the descriptive and diagnostic analyses. Predictive analysis uses the data we have summarized to make logical predictions of the outcomes of events. This analysis relies on statistical modelling, which requires added technology and manpower to forecast. It is also important to understand that forecasting is only an estimate; the accuracy of predictions relies on quality and detailed data. So, this will help us find out what is likely to happening the lot of different based predictive analysis can be performed for this particular example what we can do is to build a regression equation for region 2 sales the different factors that would affect the sales of region 2 this would help us change the different factors say for example, the advertisement expense or discount and see what would be the impact of the changes on sales so these are some things that you can do using predictive analysis. Now that we know what has happened, why it has happened and what is likely to happen our next job is to move into prescriptive analytics there, you suggest the best course of action to solve the problem..

[Audio] Prescriptive analytics is the process of using data to determine an optimal course of action. By considering all relevant factors, this type of analysis yields recommendations for next steps. Because of this, prescriptive analytics is a valuable tool for data-driven decision-making. Prescriptive analysis is the frontier of data analysis. It combines the insight from all previous analysis to determine the course of action to take in a current problem or decision. For this particular example we can give solutions like increase the advertising expense by 10% or give a 5% discount for two months if we are able to find out using the regression equation that these factors like advertising expense or discount has an impact on sales..

[Audio] So these are the four different levels of business analysis now let's summarize what we have just seen there are four different levels of business analysis the first level is descriptive analysis which will give you information on what is happening the second one diagnostic analysis which will give you information on why is it happening third one predictive analysis which would give you information on what is likely to happen in the future and the fourth level which suggests you the best course of action to solve the business problem..

[Audio] Now we will understand data modelling, various data models and data modelling techniques. Along with this we will also see data modelling process. So lets begin.

[Audio] Data modeling is the process of creating data model for the data to be stored in a database this data model is a conceptual representation of data objects the association between different data objects and the rules. The goal is to illustrate The types of data used and stored within the system. The relationships among these data types. The ways the data can be grouped and organized and its formats and attributes. After discussing this let's discuss the data model. The data model is defined as an model that organizes data description data semantics and consistency constraint of data the data model emphasizes on what data is needed and how it should be organized instead of what operations will beperformed on the data data model is like an architect's building plan which helps to build conceptual models and set a relationship between data items. Now lets understand why data modelling is that much important.

[Audio] Data modeling helps in the visual representation of data and enforces business rules regulatory compliances and government policies on the data data models ensure consistency in naming convention default values semantics security while ensuring quality of thedatathese are the main reasons that it's so important Also while fetching data from the different data sources you also design your data model if youare working in any bi tool or any other data visualization tool or any database you need to do your data modeling so that you can get the data from the different entities..

[Audio] Let's know about main types of data models widely used. Basically, there are three types of data models. Conceptual data model, logical data model and physical data model..

[Audio] Conceptual data model defines WHAT the system contains. This model is typically created by Business stakeholders and Data Architects. The purpose is to organize, scope and define business concepts and rules. Lets move on to Characteristics of a conceptual data model: Offers Organization-wide coverage of the business concepts. • This type of Data Models are designed and developed for a business audience. The conceptual model is developed independently of hardware specifications like data storage capacity location or software specifications like DBMS vendor and technology. The focus is to represent data as a user will see it in the "real world." Here you should note a very important point that conceptual data models known as domain models create a common vocabulary for all stakeholders by establishing basic concepts and scope. For example here Customer and Product are two entities. Customer number and name are attributes of the Customer entity Product name and price are attributes of product entity Sale is the relationship between the customer and product.

[Audio] Once a conceptual data model is finished, it can be used to create a less- logical one. Logical data models show how data entities are related and describe the data from a technical perspective. The logical data model is used to define the structure of data elements and to set relationships between them. The logical data model adds further information to the conceptual data model elements. Here you will see the table customer and product and there is also certain characteristics for this logical data model at this data modeling level no primary or secondary key is defined at this data modeling level you need to verify and adjust the connector details that was set earlier for relationships. Lets talk about Characteristics of a logical data model: Describes the data needs of a single project but could integrate with other logical data models based on the scope of the project. Designed and developed independently from DBMS. Data attributes will have data types with exact precisions and length..

[Audio] A logical model serves as the basis for the creation of a physical data model. A physical data model describes a database-specific implementation of the data model. It offers database ion and helps generate the schema. This Data Model describes HOW the system will be implemented using a specific DBMS system. This model is typically created by DBA and developers. The purpose is actual implementation of the database. The physical data model also helps in visualizing database structure by replicating database column keys constraints indexes triggers and other rdbms features here you define your primary and secondary key as well. Lets discuss Characteristics of a Physical data model: The physical data model describes data need for a single project or application though it maybe integrated with other physical data models based on project scope. Developed for a specific version of a DBMS, location, data storage or technology to be used in the project. Columns should have exact datatypes, lengths assigned and default values. Primary and Foreign keys, views, indexes, access profiles, and authorizations, etc. are defined..

[Audio] Let's move on to various data modelling techniques. Generally, there are six data modelling techniques which we will understand in detail..

[Audio] First is Hierarchical data modelling. Hierarchical data models organize data in a treelike arrangement of parent and child records. A child record can have only one parent, making this a one-to-many modelling method. The Hierarchical model is inspired from tree-based data structure format. Where in there is a single root node and other data is linked to the same and expands like a tree. A child node data may only have a single parent node For any parent node one to many relationship must be maintained..

[Audio] Now we will understand Network data modelling. This was also a popular data modelling option in mainframe databases that isn't used as much now. Network data models expanded on hierarchical ones by allowing child records to be connected to multiple parent records. Network model was made to overcome the drawbacks of hierarchical model. This works more like a graph rather than a tree. As the name suggests there is network based looped relationship between various data linked to one another. A child node may have more than one parent. There can be many to many relationships between data..

[Audio] Next modelling technique is relational data modelling. The relational data model was created as a more flexible alternative to hierarchical and network ones. This is the most used model and even with DBMS that we will studying in future posts it will be based upon the same. Relational model is a tabular 2-dimensional relationship between various related tables having at least one common field. This works around Rows Entities Columns Table.

[Audio] Next is Entity relationship data modelling. We will know about it in more detail further also but first have a look. It is variation of the relational model that can also be used with other types of databases, entity-relationship ( ER) models visually map entities, their attributes and the relationships between different entities. For example, the attributes of an employee data entity could include last name, first name, years employed and other relevant data. ER models provide an efficient approach for data capture and update processes, making them particularly suitable for transaction processing applications..