Overview of Business intelligence & Analytics

Overview of Business intelligence & Analytics.

[Audio] It is no surprise that Business intelligence and business analytics are two of the fastest growing market domains in the world. Organization are generating data at a rapid rate. There is a need to use this business data and make smarter decisions, companies are looking for methods and tools to turn business data into actionable insights. This is where business intelligence and analytics plays a critical role. Therefore, it becomes very important to understand and learn about these two domains..

[Audio] Hi Everyone I welcome you to this session on Business Intelligence and analytics course which contain everything you need to know. Now before we move any further let's take a look at the agenda. The first module which is introduction to Business Intelligence and business analytics will help you to understand what is Business intelligence and analytics, its significance and why it is useful. Along with this we will also cover Data, Data Analysis, Data modelling and dig down to the grass root level. We will also explore Entity relationship diagram where we will go through entities, attributes ad various relations. At the end we will also learn about data integration including ETL and ELT. This will help you to get familiar with various concepts and fundamentals related to this domain. So, lets begin..

[Audio] What is Business Intelligence Business Intelligence is a group of tools, methods, and programs that are designed to collect, analyze and transform data into valuable information ( insights) that companies can use to make better decisions and optimize their businesses It is a suite of software and services to transform data into actionable intelligence and knowledge. Business Intelligence has direct impact on organization's strategic, tactical and operational business decisions. It supports fact-based decision making using historical data rather than assumptions and gut feeling. Business intelligence uncovers insights for making strategic decisions. Business intelligence tools analyze historical and current data and present findings, in intuitive visual formats. Now we will understand what is analytics?.

[Audio] Analytics Analytics is the process of discovering, interpreting, and communicating significant patterns in data. Simply, analytics helps us see insights and meaningful data that we might not otherwise detect. Basically, analytics is the science of analyzing raw data to make conclusions about that information. Analytics provides us with meaningful information which may be hidden from us within large quantities of data. It is something that any leader, manager or just about anyone can make use of, especially in today's data-driven world. Information has long been considered as a great weapon, and analytics is the forge that creates it. Analytics changes everything, not just in the world of business, but also in science, sports, health care and just about any field where vast amounts of data are collected..

[Audio] Although business intelligence is utilized in different ways and for different purposes by individual companies, but the process is uniform throughout all industries and typically unfolds as follows: It starts with Data sourcing Data from various sources – including internal company data and external market data – is collected, integrated, and then stored. Because " Business intelligence g data" is generally used, data is commonly stored in a data warehouse, created by a data engineer. Then Data sets are created and prepared for data analysis, often by creating data analysis models. Data analysts run queries against the data sets or models. The results of queries are used to produce visualizations in the form of charts, graphs, histograms, or other visual representations, along with Business intelligence dashboards and reports to understand the criticalness of the situation and risks associated with it. Decision-makers utilize the data visualizations and reports to help stakeholders and responsible authorities in making decisions. They may also use their Business intelligence dashboard to probe further into the data for more information. Basically, it is a journey from raw data to converting it in a useful, structured and organized information. This information then generates contextual, synthesized knowledge which ultimately enables us with wisdom of understanding, integrating and taking actionable futuristic decisions..

[Audio] Although Business intelligence is all about how precisely you can use it for excavating values and benefits but superficially, we can consider the following benefits for reference. First is Implementation in any department or industry A business executive or manager might need access to data across the company, including marketing and sales, that will help him/her to make a better, more informed decision. The possibility that a business intelligence solution has an offer focuses on implementation in any industry, any function, and any data source your company might have or need. That means that a comprehensive overview can be obtained by simply implementing one of the best Business intelligence tools for your concrete business case. Further next, it Includes artificial intelligence and predictive analytics As we all know it, managing large volumes of data on a daily basis has become a Business intelligence g challenge. The goal of a smart business intelligence solution is to automate and quickly perform analysis of enormous datasets that will empower users to focus on what truly matters that is actionable insights. That means, traditional means of collecting and analyzing data such as spreadsheets no longer bring added value, but challenge organizations in their time management and costs allocation often much higher than planned. There is simply too much complex data to analyze and, consequently companies must turn to modern business intelligence system solutions that have Artificial intelligence and predictive analytics on offer. It Provides real-time monitoring and access to data, anywhere, anytime. Turning to Business intelligence means many standard, tedious processes such as exporting numerous spreadsheets can be automated through creating powerful dashboards accompanied by a number of interactive charts that you can adjust and incorporate with just a few clicks. These dashboards provide access to real-time data, refresh automatically, and have interactivity in nature to enable users to simply click on a part of the dashboard, and explore the data further. Next benefit is Designed for both data analysts and average business users Modern self-service Business intelligence tools provide options tailored for average business users, data analysts, or advanced data scientists. They need to capture different business modes that will enable each employee, no matter the background, to be able to fully comprehend and create a data story, oftentimes visualized in a striking online dashboard..

[Audio] With the help of Business Intelligence and analytics organizations these days dig down into their data. Through that analyzed data they are able to find any event which has happened in the past and along with this how that event is affecting the present. These days because of continuous Business intelligence technical advancements, reasons behind occurrence of any past or present event can be identified easily. Modern Business intelligence and analytics tools can predict and forecast upcoming events or uncertainties, which may not be precise but accurate up to an extent..

[Audio] Although business intelligence and business analytics works in simulation with each other but their core is little different. The first difference between the two concepts is in the actual definition. While both of them serve a purpose in the analytical process, they don't serve the same one. A fundamental differentiation factor is in the method each of them uses as a base. While Business intelligence tells you, what has happened in the past and what is happening now, Business Analytics tells you what will happen in the future. Usage is another factor that can help us understand how Business intelligence and Business analytics differ from each other. In this case, not only the end-user changes but also the purpose of use. As we mentioned Business intelligence tools are used with the purpose of reporting on the current and past performance of the organization, Business analytics tools allow you to take it a step further by helping you decide on your next steps. Our third and last differentiating factor is in the application. Your data is used differently depending on whether you are conducting Business intelligence or Business analytics. While Business intelligence arranges the information in easy-to-understand reports, Business analytics takes it a bit further than reporting..

[Audio] As we have understood business intelligence and analytics, now let's move on to Data first. Because without data, these processes did not even exist. Here we will cover data and its types along with various types of data analysis..

[Audio] Data is raw piece of information that is capable of being moved and store. It can be in the form of numbers, words, pictures, digits etc. In the technical era data plays a significant role to compete in the market. Many multinational companies exist around the globe which are only storing data and providing it to others. By this only they are minting huge amount of money. We as an individual user generates ample amount of data at every moment. From searching something on google to clicking on any website link, sending messages, using social media and calling also, at every instant data is being generated and stored in multiple data bases. As an individual we are generating this much data, just imagine how much data is being generated in the world at every second. This is the power of data. Data provides us with the facts. By organizing this data in different ways, we can make it easier to read, answer questions and solve problems..

[Audio] Basically, there are three types of data based on their format. Unstructured Data Semi structured data and Structured data Lets discuss these in detail..

[Audio] Unstructured data is defined as data present in absolute raw form. This data is difficult to process due to its complex arrangement and formatting. Unstructured data management may take data from many forms, including social media posts, chats, satellite imagery, IoT sensor data, emails, and presentations. to organize it in a logical, predefined manner in a data storage. Unstructured data is a data which is not organized in a predefined manner or does not have a predefined data model; thus, it is not a good fit for a mainstream relational database. For Example: Word files, PDF, Text and Media logs. Next is semi structured data..

[Audio] Semi-structured data is a type of data that has some consistent and definite characteristics. It does not confine into a rigid structure such as that needed for relational databases. Semi-structured data is information that does not reside in a relational database but that has some organizational properties that make it easier to analyze. As an Example XML files data can be considered. Now lets look structured data which is widely used..

[Audio] Structured data is information that has been formatted and transformed into a well-defined data model. It concerns all data which can be stored in database in a table with rows and columns. They have relational keys and can easily be mapped into pre-designed fields. Today, those data are most processed in the development and simplest way to manage information. Most suitable Example: is Relational data. Now let's discuss what is data analysis..

[Audio] Data analysis is defined as a process of cleaning, transforming, and modelling data to discover useful information for business decision-making. The purpose of Data analysis is to extract useful information from data and taking the decision based upon the insights. As it is clearly visible in the data analysis process diagram, it has five important stages. First is to identify the data which you want for analysis, or which is mandatory to overcome your business problem. Now, as you have identified the data second stage is to collect the data from various sources. It can be past company data, survey data etc. The collected data may have some error or not be in proper format. For eliminating these errors, it should be cleaned before analysis. Data cleaning is done through query languages. After data cleaning, the data can be analysed as per need to find insights. The findings and insights from analysis need to be interpreted properly to solve the business problem. Because whole data analysis process will not be of any worth if it is not interpreted properly..

[Audio] Now we will see important types of data analysis which is Descriptive analysis Diagnostic analysis Predictive analysis and Prescriptive analysis..

[Audio] The first type of data analysis is descriptive analysis. It is at the foundation of all data insight. It is the simplest and most common use of data in business today. Descriptive analysis answers the "what happened" or "what is happening" by summarizing past data, usually in the form of dashboards. Descriptive analytics gives you information on what is happening let us see a particular example. This report gives us information about sales of ice cream X in different regions and it's also compared these sales for years 2015 and 16. This is a typical descriptive analysis support which gives you information on what is happening to sales of this ice cream, so this chart could infer that except for region 2, the sales have increased from 2015 to 16. So, this is the level of information one could obtain from descriptive analytics. Now we know the sales have dropped in region 2, our next job is to move to the next level of analytics which is diagnostic analytics. Lets move on to diagnostic analysis..

[Audio] Diagnostic analysis takes the insights found from descriptive analytics and drills down to find the causes of those outcomes. Organizations make use of this type of analytics as it creates more connections between data and identifies patterns of behavior. A critical aspect of diagnostic analysis is creating detailed information. This begins give you information on why is it happening. So moving forward with example our job here is to deep dive into region 2 and see what exactly is going wrong. So this is a typical report for the market share of region 2. Sometimes one could infer that for all brands X Y and Z, the market share is going down because of the reason a new local brand has come up in 2016 which is gaining the market share. This is the level of information you will be able to obtain using diagnostic analysis. Now that we know what is happening to our sales why is it happening, our next step is to go to the next level of analysis which is called predictive analysis. Next move on to predictive analysis.

[Audio] Predictive analysis attempts to answer the question "what is likely to happen". This type of analysis utilizes previous data to make predictions about future outcomes. This type of analysis is another step up from the descriptive and diagnostic analyses. Predictive analysis uses the data we have summarized to make logical predictions of the outcomes of events. This analysis relies on statistical modelling, which requires added technology and manpower to forecast. It is also important to understand that forecasting is only an estimate. The accuracy of predictions relies on quality and detailed data. So, this will help us find out what is likely to happen. Different predictive analysis can be performed for this particular example. What we can do, is to build a regression equation for region 2 sales. The different factors that would affect the sales of region 2. This would help us change the different factors say for example, the advertisement expense or discount and see what would be the impact of the changes on sales. So, these are some things that you can do using predictive analysis. Now that we know what has happened, why it has happened and what is likely to happen our next job is to move into prescriptive analytics. There, you suggest the best course of action to solve the problem..

[Audio] Prescriptive analysis is the process of using data to determine an optimal course of action. By considering all relevant factors, this type of analysis yields recommendations for next steps. Because of this, prescriptive analysis is a valuable tool for data-driven decision-making. Prescriptive analysis is the frontier of data analysis. It combines the insight from all previous analysis to determine the course of action to take in a current problem or decision. For this particular example we can give solutions like, increase the advertising expense by 10% or give a 5% discount for two months. If we are able to find out using the regression equation that these factors like advertising expense or discount has an impact on sales or not..

[Audio] So these are the four different levels of business analysis. Here I have mentioned another example of sales of Air conditioners and you can understand it with reference to previous one. Now let's summarize it. Here what we have just seen, there are four different levels of business analysis. The first level is descriptive analysis which will give you information on what is happening. The second one diagnostic analysis which will give you information on why is it happening. Third one predictive analysis which would give you information on what is likely to happen in the future and the fourth level which suggests you the best course of action to solve the business problem..

[Audio] Further moving next, now we will understand data modelling, various data models and data modelling techniques. Along with this we will also see data modelling process. So, lets start with understanding data modelling..

[Audio] Data modeling is the process of creating data model for the data to be stored in a database this data model is a conceptual representation of data objects the association between different data objects and the rules. The goal is to illustrate The types of data used and stored within the system. The relationships among these data types and The ways the data can be grouped and organized and its formats and attributes. After discussing this let's discuss data model. The data model is defined as an model that organizes data description data semantics and consistency constraint of data. The data model emphasizes on what data is needed and how it should be organized instead of what operations will be performed on the data. Data model is like an architect's building plan which helps to build conceptual models and set a relationship between data items. Now lets understand why data modelling is that much significant..

[Audio] Data modeling helps in the visual representation of data and enforces business rules, regulatory compliances and government policies on the data. Data models ensure consistency in naming convention, default values, semantics and security, while ensuring quality of the data. These are the main reasons that it's so important Also, while fetching data from the different data sources, you also design your data model. If you are working on any Business intelligence tool or any other data visualization tool or any database, you need to do your data modeling. So that you can get the data from the different entities..

[Audio] Let's know about main types of data models widely used. Basically, there are three types of data models. Conceptual data model Logical data model and Physical data model. Firstly, we will know about Conceptual data model..

[Audio] Conceptual data model defines WHAT the system contains. This model is typically created by Business stakeholders and Data Architects. The purpose is to organize, scope and define business concepts and rules. Lets, move on to Characteristics of a conceptual data model: It offers Organization-wide coverage of the business concepts. This type of Data Models are designed and developed for a business audience. The conceptual model is developed independently of hardware specifications like, data storage capacity location or software specifications and technology. The focus is to represent data as, a user will see it in the "real world." Here you should note a very important point that, conceptual data models known as domain models, which create a common vocabulary for all stakeholders by establishing basic concepts and scope. For example, here Customer and Product are two entities and Customer number and name are attributes of the Customer entity. While Product name and Product price are attributes of product entity. Here, Sale is the relationship between the customer and product entities. Once a conceptual data model is finished, it can be used to create a less- logical one..

[Audio] Logical data models show how data entities are related and describe the data from a technical perspective. The logical data model is used to define the structure of data elements and to set relationships between them. The logical data model adds further information to the conceptual data model elements. For Example, Here you will see the table customer and product and there is also certain characteristics for this logical data model. At this data modeling level, no primary or secondary key is defined. At this data modeling level, you need to verify and adjust the connector details that was set earlier for relationships. Lets, talk about Characteristics of a logical data model: It describes the data needs of a single project but could integrate with other logical data models based on the scope of the project. Logical data model is Designed and developed independently from data base management system. In this model, Data attributes will have data types with exact precisions and length..

[Audio] A logical model serves as the basis for the creation of a physical data model. A physical data model describes a database-specific implementation of the data model. It offers database ion and helps generate the schema. This Data Model describes HOW the system will be implemented using a specific DBMS system. This model is typically created by Database administrator and developers. The purpose is actual implementation of the database. The physical data model also helps in visualizing database structure by replicating database column keys, constraints, indexes, triggers and other relational database management system features. Here you define your primary and secondary key as well. Lets discuss Characteristics of a Physical data model: The physical data model describes data need for a single project or application though it maybe integrated with other physical data models based on project scope. In this model Columns should have exact datatypes, lengths assigned and default values. And lastly Primary and Foreign keys, views, indexes, access profiles, and authorizations, etc. are defined..

[Audio] Moving further on various data modelling techniques. Generally, there are six data modelling techniques which we will understand in detail. Lets start with Hierarchical data modelling..

[Audio] Hierarchical data models organize data in a tree like arrangement of parent and child records. A child record can have only one parent, making this a one-to-many modelling method. The Hierarchical model is inspired from tree-based data structure format. Where in there is a single root node and other data is linked to the same and expands like a tree. Hierarchical data models a child node data may only have a single parent node. And For any parent node one to many relationship must be maintained. For example, here you can see how hierarchy of a department is represented. In the same manner hierarchical models are prepared..

[Audio] Now we will understand Network data modelling. This was also a popular data modelling option in mainframe databases that isn't used as much now. Network data models expanded on hierarchical ones by allowing child records to be connected to multiple parent records. Network model was made to overcome the drawbacks of hierarchical model. This works more like a graph rather than a tree. As the name suggests there is network based looped relationship between various data linked to one another. In Network data models a child node may have more than one parent and There can be many to many relationships between data. This picture shows a simple network data model. Here project 1 and 2, relate to multiple departments..

[Audio] Next modelling technique is relational data modelling. The relational data model was created as a more flexible alternative to hierarchical and network ones. This is the most used model and even with data base management system, it will be based upon the same. Relational model is a tabular 2-dimensional relationship between various related tables having at least one common field. This model works around Rows, Entities, Columns and Tables. This picture shows how table customer file and invoice data file are linked together by Customer I D. Invoice data file is again connected with line item file by Invoice number attribute and so on..

[Audio] Next is Entity relationship data modelling. We will know about it in more detail further also but first have a look. It is variation of the relational model that can also be used with other types of databases, entity-relationship models visually map entities, their attributes and the relationships between different entities. For example, the attributes of an employee data entity could include last name, first name, years employed and other relevant data. Same is shown in the picture for student and course entities with attributes Student I D, Age, Course I D and credits. Entity relationship models provide an efficient approach for data capture and update processes, making them particularly suitable for transaction processing applications..

[Audio] Next modelling technique is dimensional data modelling. Dimensional data models are primarily used in data warehouses and data marts that support business intelligence applications. They consist of fact tables that contain data about transactions or other events and dimension tables that list attributes of the entities in the fact tables. For example, a fact table could have details like product purchases by customers, while connected dimension tables hold data about the products and customers..

[Audio] Last data modelling technique is object-oriented data modelling. The object-oriented approach is like the entity relationship method in how it represents data, attributes and relationships, but it s entities into objects. Different objects that have the same attributes and behaviors can be grouped into classes, and new classes can inherit the attributes and behaviors of existing ones. But object databases remain a niche technology for applications, which has limited the use of object-oriented modelling..

[Audio] Data modelling techniques have different conventions but all approaches provide formalized workflows that include a sequence of tasks to be performed in an iterative manner. Those workflows generally look like this: First step is Identify the entities. The process of data modelling begins with the identification of the things, events or concepts that are represented in the data set that is to be modelled. Each entity should be cohesive and logically discrete from all others. Next step is Identify key properties of each entity. Each entity type can be differentiated from all others because it has one or more unique properties, called attributes. For instance, an entity called "customer" might possess such attributes as a first name, last name, telephone number and salutation, while an entity called "address" might include a street name and number, a city, state, country and zip code. Then third step is to Identify relationships among entities. The earliest draft of a data model will specify the nature of the relationships each entity has with the others. In the above example, each customer "lives at" an address. If that model were expanded to include an entity called " orders," each order would be shipped to and Business intelligence lled to an address as well. These relationships are usually documented via unified modelling language ( UML). After this next step is to Map attributes to entities completely. This will ensure the model reflects how the business will use the data. Several formal data modelling patterns are in widespread use. Object-oriented developers often apply analysis patterns or design patterns, while stakeholders from other business domains may turn to other patterns. Moving forward to next step which is Assign keys as needed and decide on a degree of normalization that balances the need to reduce redundancy with performance requirements. Normalization is a technique for organizing data models (and the databases they represent) in which numerical identifiers, called keys, are assigned to groups of data to represent relationships between them without repeating the data. For instance, if customers are each assigned a key, that key can be linked to both their address and their order history without having to repeat this information in the table of customer names. Normalization tends to reduce the amount of storage space a database will require, but it can at cost to query performance. Than at the end Finalize and validate the data model. Data modelling is an iterative process that should be repeated and refined as business needs change..

[Audio] Now we will learn about Entity Relationship diagram starting with what is an Entity Relationship diagram and why it's been so much used by the companies. Then we will learn about the symbols used in the E R diagram and get familiar with the components of it. So let's start.

[Audio] Have you ever wondered how these Business intelligence users and ecommerce companies manage their tons of data that keeps updating every second and these datasets keep on updating in their databases. To perform operations on this data they should have a conceptual understanding of these databases and this is done by understanding E R diagrams of the databases. So let's understand what an E R diagram is An entity relationship diagram describes the relationship of entities that needs to be stored in a database. E R diagram is mainly a structural design for the database. It is a framework made using specialized symbols to define the relationship between entities. E R diagrams are created based on the three main components entities, attributes and relationships. let's understand the use of E R diagram with the help of a real-world example. A school needs all its student records to be stored digitally, so they approach an IT company to do so. A person from the company will meet the school authorities to understand all their requirements and describe them in the form of E R diagram and get it close checked by the school authorities. As the school authorities approved the Entity Relationship diagram, the database engineers would carry further implementation..

[Audio] Let's have a view of an E R diagram. The following diagram showcases two entities student and course and the relationship. The relationship described between student and course is many to many as a course can be opted by several students and a student can opt for more than one course. Here student is the entity and it possesses the attributes that is student I D, student name and student age and the course entity has attributes such as course I D and course name. Now we have an understanding of E R diagram..

[Audio] Let us see why it has been so popular. The logical structure of the database provided by E R diagram communicates the landscape of business to different teams in the company which is eventually needed to support the business. ER diagram is a Graphical user interface representation of the logical structure of a database, which gives a better understanding of the information to be stored in a database. Database designers can use E R diagrams as a blueprint which reduces complexity and helps them save time to build databases quickly. E R diagrams helps you identify the entities that exist in a system and the relationships between those entities after knowing its uses. Now we should get familiar with the symbols used in E R diagram..

[Audio] The rectangle symbol represents the entities. Oval symbol represents attributes. A rectangle embedded in a rectangle represents a weak entity. A dashed oval represents a derived attribute. A diamond symbol represents a relationship among entities. Double oval symbol represents multi-valued attributes. Now we should dive in and learn about the components of Entity relationship diagram..

[Audio] There are three main components of E R diagram entity, attribute and relationship. Entities have weak entity Attributes are further classified into key attribute, composite attribute, multivalued attribute and derived attribute. Relationships are also classified into one-to-one relationships, one-to-many relationships, many-to-one relationships and many-to-many relationships. Let's understand these components of ER diagram..

[Audio] Starting with entities An entity can be either a living or a non-living component. An entity is showcased as a rectangle in an ER diagram. Let's understand this with the help of an E R diagram. Here both student and course are in rectangular shape and are called entities and they represent the relationship study in a diamond shape. Let's transition to weak entity..

[Audio] An entity that makes a lines over another entity is called a weak entity. The weak entity is showcased as a double rectangle in E R diagram. In the example below the school is a strong entity because it has a primary key attribute school number. Unlike the school the classroom is a weak entity because it does not have any primary key and the room number attribute here acts only as a discriminator and note of primary key. Now let us know about attributes..

[Audio] An attribute exhibits the properties of an entity. An attribute is illustrated with an oval shape in an E R diagram In the example below student is an entity and the properties of student such as address, age name and role number are called its attributes. Let's see our first classification under attribute.

[Audio] The key attribute uniquely identifies an entity from an entity set. The text of a key attribute is underlined. In the example, below we have a student entity and it has attributes name, address roll number and age. But here roll number can uniquely identify a student from a set of students that's why it is termed as a key attribute. Now we will see composite attribute..

[Audio] An attribute that is composed of several other attributes is known as a composite attribute and oval showcases the composite attribute. The composite attribute oval is further connected with other roles In the example below we can see an attribute name, which can have further subparts such as first name, middle name and last name. These attributes with further classification is known as composite attribute. Now let's have a look at multi-valued attribute..

[Audio] An attribute that can possess more than one value are called multi-valued attributes. These are represented as double oval shape. In the example below the student entity has attributes phone number, roll number, name and age. Out of these attributes, phone number can have more than one entry and the attribute with more than one value is called multivalued attribute. Let's see derived attribute..

[Audio] An attribute that can be derived from other attributes of the entity is known as a derived attribute. In the E R diagram the derived attribute is represented by dashed oval. and in the example below student entity has both date of Birth and age as attributes. Here age is a derived attribute, as it can be derived by subtracting current date from the student date of Birth. Now after knowing attributes let's understand relationship in E R diagram..