Unlocking the Secrets of CALCULATE: A Deep Dive into Advanced Data Analysis in Power BI


Are you tired of drowning in a sea of data? Data analysis is a puzzle waiting to be solved and CALCULATE is the missing piece that brings it all together. Let’s explore the intricacies of CALCULATE in Power BI. From unraveling complex calculations to applying complex filters, this function holds the key to unlocking actionable insights buried within your data. Whether your a business professional, a data enthusiast, or a seasoned data analyst this guide will equip you with the knowledge and tools to solve the most perplexing data puzzles. Brace yourself for a comprehensive exploration of Power BI’s CALCULATE function.

Prepare to be amazed as we explore the the secrets of the CALCULATE function. CALCULATE is the true superhero of Power BI that empowers you to perform complex calculations and transformations on your data effortlessly. It holds the key to manipulating the filter context, allowing you to focus on the precise subset of data you need for your analysis.

For those of you eager to start experimenting and diving deeper there is a sample Power BI report loaded with the sample data used in this post ready for you. So don’t just read, dive in and get hands-on with DAX Functions in Power BI. Check out it here: GitHub — Power BI DAX Function Series: Mastering Data Analysis.


Understanding the Syntax and Parameters of CALCULATE

Before we dive into the secrets of CALCULATE and explore practical examples, let’s first understand its syntax and parameters. The CALCULATE function follows a simple structure:

CALCULATE(expression, filter1, filter2, ...)

The expression parameter represents the calculation or measure you want to evaluate or modify. It can be a simple aggregation like SUM or AVERAGE or a more complex calculation involving multiple DAX functions. The filter parameters are optional and allow you to define specific conditions or constraints to modify the filter context.

Each filter parameter can take various forms, such as direct values, comparison operators, or logical expressions. You can combine multiple filters using logical operators like && (AND) or || (OR) to create more intricate filter conditions. By strategically using the filter parameters within CALCULATE, you can dynamically adjust the filter context and precisely control which data is included in your calculations.

By understanding the syntax and leveraging the flexibility of the CALCULATE parameters, you can master this powerful function and have the ability to handle complex data analysis with ease.


Leveraging the Power of CALCULATE: Practical Examples in Power BI

Calculating Total Sales for a Specific Region and Time Period

Let’s dive into the heart of CALCULATE and explore its power through various examples. Imagine you have a dataset with sales figures for various products across different regions and want to calculate the total sales for a specific region, but only for a particular time frame. By combining CALCULATE with it’s filter parameters, you can create a dynamic calculation that narrows down the data based on the desired filters. This enables you to zero in on the exact information you need and present accurate, targeted insights.

For instance, using CALCULATE you can easily calculate last year total sales of smartphones in the United States. The DAX formula is defined by the following expression:

CALCULATE(

    SUM(Sales[Amount]),

    SAMEPERIODLASTYEAR(Dates[Date]),

    Products[Product] = "Smartphone",

    Regions[Region] = "United States"
)

This expression filters the Sales table based on the specified conditions, summing up the Amount column to give you the total sales of smartphones in the United States for the previous year.

Tracking Cumulative Sales Over Time

Another powerful application of CALCULATE lies in calculating running totals or cumulative values. Let’s say you want to track cumulative sales for each month of the year. With the help of the SUM function and CALCULATE, you can easily create a measure that accumulates the sales for each month, taking into account the changing filter context. This allows you to visualize the sales growth over time and identify any notable trends and patterns.

The DAX formula for this scenario would be:

Cumulative Sales =
CALCULATE(
    SUM(Sales[Amount]),
    FILTER(
        ALL(Sales),
        Sales[SalesDate] <= MAX(Sales[SalesDate])
    )
)

This formula calculates the cumulative sales by summing up the Amount column for all Sales Dates up to and including the last Sales Date as determined by MAX(Sales[SalesDate]).

Refined Average Sales for High-Performing Regions

Conditional calculations are also a breeze with CALCULATE. Suppose you want to calculate the average sales for a specific product category, but only for the regions where sales exceed a certain threshold. By combining CALCULATE with logical filters based on sales, you can obtain a refined average that factors in only the high-performing regions. Enabling you to make data-driven decisions with confidence.

The DAX formula for this example would be:

High Performing Average =
CALCULATE(
    AVERAGE(Sales[Amount]),
    FILTER(
        ALL(Sales[RegionID]),
        CALCULATE(
            SUM(Sales[Amount])
        ) > 37500
    )
)

This formula will calculate the average sales for a product category but only considers the regions where the total sales exceed $37,500. The CALCULATE function modifies the filter context and focuses on the desired subset of data, allowing you to obtain a more refined average.


CALCULATES Versatility

CALCULATE’s true strength lies in its versatility. You can combine it with other DAX functions, such as ALL, RELATED, or TOPN to further enhance your data analysis capabilities. Whether you need to compare values against a benchmark, calculate year-to-date totals, determine the top-performing products, or even perform advanced calculations based on complex conditions. CALCULATE is the tool that will bring your data analysis to the next level.

CALCULATE introduces the concept of internal and external filters which play a crucial role in shaping the filter context for calculations. Internal filters are defined within CALCULATE itself using the filer parameters. These filters modify the filter context only for the expression being evaluated within CALCULATE. On the other hand, external filters are filters that exist outside of CALCULATE and are not affected by the function. Understanding the interplay between internal and external filters is key to harnessing the full power of CALCULATE.

Applying External Filters with CALCULATE: Comparing Performance Against a Benchmark

Let’s say you want to compare the sales of smartphones in the United States against a benchmark value, such as the average sales of smartphones across all regions. This comparison can help identity regions that are outperforming and underperforming relative benchmarks.

The DAX expression for this example would be:

US Smartphone Sales vs. Average Smartphone Sales =
    CALCULATE(
        AVERAGE(Sales[Amount]),
        Products[Product] = "Smartphone",
        Regions[Region] = "United States"
    )
    - AVERAGEX(
        FILTER(
            Sales,
            RELATED(Products[Product]) = "Smartphone"
        ),
        Sales[Amount]
)

This expression calculates the total sales of smartphones in the United States and subtracts the average sales of smartphones across all regions. The FILTER function ensures that only the relevant products (i.e. smartphones) are considered in the average calculation.

Dynamic Calculations with CALCULATE: Adjusting for Changing Contexts

Calculating year-to-date (YTD) totals is another common requirement in data analysis. To calculate YTD sales you can leverage the time intelligence functions in DAX. With CALCULATE and DATESYTD function you can easily obtain YTD sales figures.

The DAX expression would be:

YTD Sales =
CALCULATE(
    SUM(Sales[Amount]),
    DATESYTD(
        Dates[Date],
        "12/31"
    )
)

Enhancing Filter Context with KEEPFILTERS and CALCULATE

In some scenarios, you may want to preserve any existing filters on other dimensions such as date, region, or employee while using CALCULATE to introduce additional filters. This is where the KEEPFILTERS function comes into play. By wrapping your expression within KEEPFILTERS, you ensure that the existing filters remain unchanged and only the internal filters in CALCULATE are applied. This allows you to have precise control over the filter context and produce accurate results.

The DAX formula for this scenario would look like this:

Smartphone Sales =
CALCULATE(
    SUM(Sales[Amount]),
    KEEPFILTERS(Sales[ProductID]=1)
)

By applying this formula, you can obtain the accurate sales amount for the desired product type, while keeping the context of other dimension intact (e.g. Region). This enables you to perform focused analysis and make data-driven decision based on specific criteria.


Conclusion

Congratulations! You have completed the thrilling exploration of the CALCULATE function in Power BI. Through the practical examples you have witnessed its remarkable ability to manipulate the filter context, allowing you to extract meaningful insights from your data with precision. From calculating specific totals and cumulative values to comparing against benchmarks and performing complex conditional calculations, CALCULATE has proven to be a formidable tool in your data analysis arsenal. By mastering CALCULATE, you can unlock the power to transform raw data into actionable insights, enabling data-driven decisions-making.

As Albert Einstein once said, “Anyone who has never made a mistake has never tried anything new.” So, don’t be afraid of making mistakes; practice makes perfect.

Continuously experiment with CALCULATE, explore new DAX functions, and challenge yourself with real-world data scenarios.


Thank you for reading! Stay curious, and until next time, happy learning.

And, remember, as Albert Einstein once said, “Anyone who has never made a mistake has never tried anything new.” So, don’t be afraid of making mistakes, practice makes perfect. Continuously experiment and explore new DAX functions, and challenge yourself with real-world data scenarios.

If this sparked your curiosity, keep that spark alive and check back frequently. Better yet, be sure not to miss a post by subscribing! With each new post comes an opportunity to learn something new.

From Data to Insights: Maximizing Power BI’s Calculated Measures and Columns for Deeper Analysis


Explore the power of calculated measures and columns in Power BI that will revolutionize the way you uncover insights and elevate your Power BI skills.

Introduction

In the world of data analysis, the ability to derive meaningful insights from raw data is crucial. Power BI empowers you to go beyond just the basics and unlock the full potential of your data through calculated measures and columns. These game-changing features allow you to perform complex calculations and create new data points based on existing information. Enabling you to gain deeper insights and make more and better informed decisions.


Calculated Measures

Creating calculated measures in Power BI is a straightforward process. With just a few simple steps, you can unleash a whole new level of analysis. For example, say you have a sales dataset and want to calculate the average unit price of products sold. This can easily be accomplished by creating a calculated measure.

Start by opening Power BI Desktop and navigating to the report or dataset where you want to create the calculated measure. Right-click on the desired table, select New Measure and enter the required formula or expression that defines the calculation. To demonstrate the example above we will enter:

Average Unit Price = AVERAGE(SalesOrderDetail[UnitPrice])

Power BI will instantly calculate the average unit price based on the defined formula.

But wait, there is more! Calculated measures go way beyond just calculating basic aggregations. We can step up our calculated measure game by using DAX iterator functions. Iterator functions are DAX expressions that operate row-by-row, or are in Power BI referred to as having row context. These functions typically end with an X (e.g. SUMX, AVERAGEX).

Our Sales table has OrderQty, UnitPrice and UnitPriceDiscount columns but no column for the sales amount. We are interested in this sales amount value and how it trend over time.

To analyze this we can create a new measure Sales Amount defined by the following expression:

SalesAmount = 
SUMX(
   SalesOrderDetail, SalesOrderDetail[OrderQty] * 
   SalesOrderDetail[UnitPrice] * (1 - 
   SalesOrderDetail[UnitPriceDiscount])
)

This calculated measure allows you to gain insights into the overall sales performance and identify patterns or trends over time.

For a deep dive and further exploration of iterator functions check out the Power BI Iterators: Unleashing the Power of Iteration in Power BI Calculations.

Iterator Functions — What they are and What they do

Whether it’s aggregating data, calculating ratios, or applying logical functions, Power BI offers a rich set of DAX functions that have got you covered.


Calculated Columns

In addition to calculated measures Power BI also offers the ability to create calculated columns. Calculated column take your data analysis to another level by allowing you to create new data points at the individual row level. The possibilities are endless when you can combine existing columns, apply conditional logic, or generate dynamic values. Let’s consider a products dataset where you have the Product Number which contains a two letter product type code followed by the product number. For your analysis you require an additional column containing just the product type identifier. Calculated columns are well suited to meet this need.

Within Power BI Desktop right-click the table where you want to add the calculated column, select New Column, and define the formula or expression. To extract the first two characters (i.e. the product type code) we will use:

ProductType = LEFT(Products[ProductNumber],2)

Power BI will extract the first two characters of the Product Number for each row, creating a new Product Type column. This calculated column makes it easier to analyze and filter data based on the product type. Power BI’s intuitive interface and DAX language make this process seamless and approachable.

For more details on calculated measures and columns check out Power BI Row Context: Understanding the Power of Context in Calculations.

Row Context — What it is, When is it available, and its Implications

The post highlights the key differences between calculated measures and columns and when it is best practice or beneficial to you one method over the other.


Beyond the Basics

Why limit yourself to basic calculations? Power BI’s calculated measures and columns give you the power to dig deeper into your data. Using complex calculations you can uncover deeper patterns, trends, and correlations that were previously hidden. For example, with our sales data we want to analyze the percentage of total sales for each product type. With Power BI, you can create a calculated measure using the formula:

Percentage Sales =
VAR Sales = SalesOrderDetail[Sales Amount]
VAR AllSales = 
  CALCULATE(
    SalesOrderDetail[Sales Amount], 
    REMOVEFILTERS(Products[Product Type])
  )

RETURN
DIVIDE(Sales, AllSales)

Important Concepts

To continue to elevate your skills in developing calculated measures and columns there are a few concepts to understand. These include row context, filter context, and context transition.

When Power BI evaluates DAX expressions the values the expression can access are limited by what is referred to as the evaluation context. The two fundamental types of evaluation context are row context and filter context.

For further information and a deeper dive into these concepts checkout the following posts:

Row Context — What it is, When is it available, and its Implications

Filter Context – How to create it and its impact on measures

Remember, the flexibility of calculated measures and columns in Power BI allows you to customize and adapt your calculations to suit your specific business needs. With a few simple and well crafted formulas, you can transform your data into meaningful insights and drive data-informed decisions.


Visualize Your Calculations

By incorporating calculated measures and columns into you visualizations you can communicate your data insights effectively. Drag and drop these calculations into your reports and dashboards to display dynamic results that update in real-time. Combine them with filters, slicers, and interactive features to empower users to explore the data and gain deeper insights on their own.


Conclusion

With the power of calculated measures and columns in Power BI, you have the tools to elevate your data analysis to new heights. Discover the full potential of your data, uncover hidden insights, and make data-driven decisions with confidence. Embrace the simplicity and versatility of calculated measures and columns in Power BI and watch your data analysis thrive. Get ready to embark on your journey to deeper insights and unlock the true power of your data.


Thank you for reading! Stay curious, and until next time, happy learning.

And, remember, as Albert Einstein once said, “Anyone who has never made a mistake has never tried anything new.” So, don’t be afraid of making mistakes, practice makes perfect. Continuously experiment and explore new DAX functions, and challenge yourself with real-world data scenarios.

If this sparked your curiosity, keep that spark alive and check back frequently. Better yet, be sure not to miss a post by subscribing! With each new post comes an opportunity to learn something new.

Unleashing the Power of Power Query: A Guide to Power Query in Power BI

Introduction

Power Query is a powerful data preparation and transformation tool within Power BI. It empowers users to connect to various data sources, shape and cleanse data, and load it into the Power BI environment for visualization and analysis.

This blog post will explore what Power Query is, the ins and outs of Power Query and how to use it effectively leveraging its full potential in Power BI for data analysis.

What is Power Query

Power Query is a versatile data connectivity and transformation tool that enables users to extract, manipulate, and load data from a wide range of sources into Power BI. It provides an intuitive user interface providing a comprehensive set of data preparation functionalities. The data preparation tools help transform raw messy data into clean data suitable for analysis.


How to use Power Query

Lets explore how to leverage Power Query to retrieve data from data sources and perform transformations to prepare data for analysis.

Connecting to Data Sources

We can access Power Query from Power BI Desktop. On the top ribbon click the “Get Data” button on the Home tab. Selecting the chevron will show a list of common data sources, to view all data sources select more listed on the bottom or you can select the icon above “Get Data”.

Choose the desired data sources from the available options. Available sources include databases, Excel files, CSV files, web pages, and cloud-based services. Provide the required credentials and connection details to establish a connection to the selected data sources.

Data Transformation and Cleansing

Power Query provides a range of data transformation capabilities. Utilizing the Power Query Editor you can shape and clean data to meet your requirements. You can perform operations like filtering, sorting, removing duplicates, splitting columns, renaming columns, merging data from multiple sources and creating custom calculated columns.

Filter and sorting data using a familiar interface.

Remove, split, and rename columns within your dataset.

Ensure the correct data types of you data by setting the column data type.

Leverage the power of Power Query functions and formulas to optimize your data transformation process.

Applied Steps

As you build your transformation Power Query using either built-in functions or custom transformations using the Power Query Formula Language (M Language) each transformation is recorded as an Applied Step. Each Applied Step can be viewed in the Query Settings panes.

You can review and modify the Applied Steps to adjust the data transformation process as required. During the review of the Applied Steps you can further refine the data preparation process and improve query performance. Implementing query folding and other query optimization techniques can improve the efficiency of the your Power Queries.

Query Dependencies and Data Merging

Power Query enables the the development of multiple queries, each representing a specific data source or data transformation step. You can utilize query dependencies to define relationships between queries, allowing for data merging and consolidation. Leverage merging capabilities to combine data from multiple queries based on common fields, such as performing inner joins, left joins, or appending data.

Combine or merge data from multiple queries based on one or more matching column with the Merge Queries operation.

Proper use of merging capabilities can optimize your data analysis process.

Query Parameters, Dynamic Filtering, and Functions

Power Query allows for the use of query parameters. These query parameters act as placeholder for values that can be dynamically changed. This allows for dynamic filtering options. The use of query parameters can increase the flexibility, interactivity, and reusability of queries and the resulting Power BI reports.

Custom functions within Power Query can be used to encapsulate complex data transformations and you can reuse them across multiple queries.

Data Loading and Refreshing

After applying the required transformations, you load the data into the Power BI data model by clicking Close & Apply. Power Query creates a new query or appends the transformed data to an existing query within the Power BI data model. To ensure the data stays up to date with the source systems by setting up automatic data refreshes.

Advanced Power Query Features

There are advanced features within Power Query such as conditional transformations, grouping and aggregation, unpivoting columns, and handling advanced data types. These features and other optimization techniques can be implemented to handle complex data scenarios and improve efficiency of you data analysis.


Conclusion

Power Query is a powerful tool for data preparation and transformation in Power BI. Its approachable interface and expansive capabilities empower users to connect to various data sources, cleanse and shape data, and load it into the Power BI data model. By expanding your knowledge and use of Power Query advanced features you can optimize your data analysis process, streamline data preparation, and unlock the full potential of your data. Implement the strategies outlined in this guide to improve your Power BI reports and dashboards expanding your analysis to new heights of insight and effectiveness.

Start your exploration of Power Query and its features to further the effectiveness of your data analysis with increased flexibility and efficiency.


Thank you for reading! Stay curious, and until next time, happy learning.

And, remember, as Albert Einstein once said, “Anyone who has never made a mistake has never tried anything new.” So, don’t be afraid of making mistakes, practice makes perfect. Continuously experiment and explore new DAX functions, and challenge yourself with real-world data scenarios.

If this sparked your curiosity, keep that spark alive and check back frequently. Better yet, be sure not to miss a post by subscribing! With each new post comes an opportunity to learn something new.

Streamline Application Lifecycle Management with Power Platform Solutions

Application lifecycle management (ALM) is a critical process for any software development projects. ALM is a comprehensive process for developing, deploying, and maintaining robust scalable applications. Power Platform solutions offer a powerful toolkit that enables application development and provide essential ALM capabilities. ALM plays a vital role in maximizing the potential and ensuring the success of you Power Platform solutions.

The Microsoft Power Platform is a comprehensive suite of tools that includes Power Apps, Power Automate, and Power BI. These tools empower organizations to create custom applications, automate processes, and gain insights from data. Implementing ALM practices within the Power Platform can streamline the development process and deliver high-quality applications.

This blog post will explore how implementing ALM practices can enhance collaboration, improve performance, and streamline development for Power Platform solutions.


Why you need ALM

ALM for Power Platform solutions is crucial for several reasons:

Ensuring Quality, Security, and Performance: ALM practices help organizations maintain the quality, security, and performance of their applications across different environments. It ensures that applications meet the desired standards and perform optimally.

Collaborating with Other App Makers: ALM enables seamless collaboration between app makers within an organization. It provides a consistent development process, allowing multiple stakeholders to work together effectively.

Managing Dependencies and Compatibility: Power Platform solutions consist of various components such as tables, columns, apps, flows, and chatbots. ALM helps manage dependencies between these components and ensures compatibility across different versions and environments.

Automating Deployment and Testing: ALM enables organizations to automate the deployment and testing of Power Platform applications. It simplifies the process of tracking changes, applying updates, and ensuring the reliability of applications.

Monitoring and Collecting Feedback: ALM practices facilitate monitoring and troubleshooting of applications. They enable organizations to collect feedback from end-users, identify issues, and make necessary improvements.


How to implement ALM

To implement ALM for Power Platform solutions, building projects within solutions is essential. Solutions serve as containers for packaging and distributing application artifacts across environments. They encompass all the components of an application, such as tables, columns, apps, flows, and chatbots. Solutions can be exported, imported, and used to apply customizations to existing apps.

Collaborative Development

The Power Platform’s low-code development platform provides a collaborative environment for creators, business users, and IT professionals. The platform includes features like solution management and environment provisioning which play a role in establishing ALM for your Power Platform projects. The solution explorer enables managing multiple app components, tracking changes, and merging code updates. By enabling collaborative development, the Power Platform encourages teamwork and reduces conflicts during the development lifecycle.

Version Control and Change Management

When collaborating on components of a solution source control can be used as the single source of truth for storing each component. Source control is a system that tracks the changes and version of your code and allows you to revert or merge them as required.

Version control and change management are crucial elements of ALM. They ensure an organized development process and enable efficient management of code changes. The Power Platform integrates with source control tools such as GitHub or Azure DevOps, allowing developers to track changes, manage branches, and merge code updates effectively. Incorporating version control and change management practices allows you to establish a robust foundation for ALM.

Testing and Quality Assurance

Testing is a crucial phase in the ALM process to ensure the reliability and quality of Power Platform applications. The Power Platform provides various testing options to validate your solutions. Power Apps allows for unit testing, where developers can create and run automated tests to validate app functionality. Power Automate offers visual validation and step-by-step debugging for workflows. Power BI allows the creation of test datasets and simulation of real-world scenarios. Comprehensive testing practices identify and resolve issues early, ensuring the delivery of high-quality applications.

Continuous Integration and Deployment

Integrating Power Platform solutions with tools like Azure DevOps and GitHub enables continuous integration and deployment (CI/CD) pipelines. These automation tools streamline the deployment and testing processes. For example, Azure DevOps provides automation and release management capabilities allowing you to automate the deployment of Power Apps, Power Automate flows, and Power BI reports. With CI/CD pipelines, organizations can automate the build, testing, and deployment of their solutions. This approach accelerates release time, reduces human errors, and maintains consistency across environments. CI/CD pipelines also promote Agile and DevOps methodologies, fostering a culture of continuous improvement.

Monitoring and Performance Optimization

Once your applications are deployed, monitoring and performance optimization become an essential aspect of ALM. Monitoring tools can help you identify and resolve issues with your applications and improve their quality and functionality. Power Platform solutions provide built-in monitoring capabilities and integrate with Azure Monitor and Applications Insights. These tools offer real-time monitoring, performance analytics, and proactive alerts. Leveraging these features helps organizations identify and address issues promptly, optimize performance, and deliver a seamless end-user experience.


Conclusion

The Microsoft Power Platform offers a rapid low- to no-code platform for application and solution development. However, incorporating ALM practices goes beyond rapid development. By leveraging Power Apps, Power Automate, Power BI, and their integration with tools like Azure DevOps and GitHub, organizations can streamline collaborative development, version control, testing, and deployment. Implementing ALM best practices ensures the delivery of high-quality applications, efficient teamwork, and continuous improvement. Embracing ALM in Power Platform solutions empowers organizations to develop, deploy, and maintain applications with agility and confidence.

Now its time to maximize the potential of you Power Platform solutions by implementing ALM practices.


Thank you for reading! Stay curious, and until next time, happy learning.

And, remember, as Albert Einstein once said, “Anyone who has never made a mistake has never tried anything new.” So, don’t be afraid of making mistakes, practice makes perfect. Continuously experiment and explore new DAX functions, and challenge yourself with real-world data scenarios.

If this sparked your curiosity, keep that spark alive and check back frequently. Better yet, be sure not to miss a post by subscribing! With each new post comes an opportunity to learn something new.

Power BI — Key Differences Between Data Connectivity Modes

Introduction

Power BI is a data analysis and reporting tool that connects to and consumes data from a wide variety of data sources. Once connected to data sources it provides a power tool for data modeling, data visualization, and report sharing.

All data analysis projects start with first understanding the business requirements and the data sources available. Once determined the focus shifts to data consumption. Or how to load the required data into the analysis solution to provide the required insights.

Part of dataset planning is determining between the various data Power BI connectivity modes. The connectivity modes are methods to connect to or load data from the data sources. The connectivity mode defines how to get the data from the data sources. The selected connectivity mode impacts report performance, data freshness, and the Power BI features available.

The decision is between the default Import connectivity mode, DirectQuery connectivity mode, Live Connection connectivity mode, or using a composite model. This decision can be simple in some projects, where one option is the only workable option due to the requirements. In other projects this decision requires an analysis of the benefits and limitations of each connectivity mode.

So which one is the best?
Well…it depends.

Each connectivity type has its use cases and generally one is not better than the other but rather a trade-off decision. When determining which connectivity mode to use it is a balance between the requirements of the report and the limitations of each method.

This article will cover each connectivity mode and provides an overview of each method as well as covering their limitations.


Overview

Import mode makes an entire copy of a subset of the source data. This data is then stored in-memory and made available to Power BI. DirectQuery does not load a copy of the data into Power BI. Rather Power BI stores information about the schema or shape of the data. Power BI then queries the data source making the underlying data available to the Power BI report. Live Connection store a connection string to the underlying analysis services and leverages Power BI as a visualization layer.

As mentioned in some projects determining a connectivity mode can be straight forward. In general, when a data source is not equipped to handle a high volume of analytical queries the preferred connectivity mode is Import mode. When there is a need for near real-time data then DirectQuery or Live Connection are the only options that can meet this need.

For the projects where you must analyze each connectivity mode, keep reading to get a further understanding of the benefits and limitations of each mode.


Getting Started

When establishing a connection to a data source in Power BI you are presented with different connectivity options. The options available depend on the selected data source.

Available data sources can viewed by selecting Get data on the top ribbon in Power BI Desktop. Power BI presents a list of common data sources, and the complete list can be viewed by selecting More... on the bottom of the list.

Loading or connecting to data can be a different process depending on the selected source. For example, loading a local Excel file you are presented with the following screen.

From here you can load the data as it is within the Excel file, or Transform your data within the Power Query Editor. Both options import a copy of the data to the Power BI data model.

However, if the SQL Server data source is select you will see a screen similar to that below.

Here you will notice you have an option to select the connectivity mode. There are also additional options under Advanced option such as providing a SQL statement to evaluate against the data source.

Lastly, below is an example of what you are presented if you select Analysis Services as a data source.

Again here you will see the option to set the connectivity mode.


Import

Import data is a common, and default, option for loading data into Power BI. When using Import Power BI extracts the data from the data source and stores it in an in-memory storage engine. When possible it is generally recommended to use Import mode. Import mode takes advantage of the high-performance query engine, creates highly interactive reports offering the full range of Power BI features. The alternative connectivity modes discussed later in this article can be used if the requirements of the report cannot be met due to the limitations of the Import connectivity mode.

Import models store data using Power BI’s column-based storage engine. This storage method differs from row-based storage typical of relational database systems. Row-based storage commonly used by transactional systems work well when the system frequently reads and writes individual or small groups of rows.

However, this type of storage does not perform well with analytical workloads generally needed for BI reporting solutions. Analytical queries and aggregations involve a few columns of the underlying data. The need to efficiently execute these type of queries led to the column-based storage engines which store data in columns instead of rows. Column-based storage is optimized to perform aggregates and filtering on columns of data without having to retrieve the entire row from the data source.

Key considerations when using the Import connectivity mode include:

1) Does the imported data needed to get update?

2) How frequent does the data have to be updated?

3) How much data is there?

Import Considerations

  • Data size: when using Power BI Pro your dataset limit is 1GB of compressed data. With Premium licenses this limit increases to 10GB or larger.
  • Data freshness: when using Power BI Pro you are able to schedule up to 8 refreshes per day. With Premium licenses this increases to 48 or every 30 minutes.
  • Duplicate Work: when using analysis services all the data modeling may already be complete. However, when using Import mode much of the data modeling may have to be redone.

DirectQuery

DirectQuery connectivity mode provides a method to directly connect to a data source so there is no data imported or copied into the Power BI dataset. DirectQuery can address some of the limitations of Import mode. For example for a large datasets the queries are processed on the source server rather than the local computer running Power BI Desktop. Additionally since it provides a direct connection there is less of a need for data refreshes in Power BI. DirectQuery report queries are ran when the report is opened or interacted with by the end user.

Like importing data, when using DirectQuery with an on-premises data source an on-premises data gateway is required. Although there is no schedule refreshes when using DirectQuery the gateway is still required to push the data to the Power BI Service.

While DirectQuery can address the limitations presented by Import mode, DirectQuery comes with its own set of limitations to consider. DirectQuery is a workable option when the underlying data source can support interactive query results within an acceptable time and the source system can handle the generated query load. Since with DirectQuery analytical queries are sent to the underlying data source the performance of the data source is a major consideration when using DirectQuery.

DirectQuery Consideration

  • A key limitation when considering DirectQuery is that not all data sources available in Power BI support DirectQuery.
  • If there are changes to the data source the report must be refreshed to show the updated data. Power BI reports use caches of data and due to this there is no guarantee that visuals always display the most recent data. Selecting Refresh on the report will clear any caches and refresh all visuals on the page.

Example
Below is an example of the above limitation. Here we have a Power BI table and card visual of a products table on the left and the underlying SQL database on the right. For this example we will focus on ProductID = 680 (HL Road Frame – Black, 58) with an initial ListPrice of $1,431.50.

The ListPrice is updated to a value of $150,000 in the data source. However, after the update neither the table visual nor the card showing the sum of all list prices updates.

There is generally no change detection or live streaming of the data when using DirectQuery.

When we set the ProductID slicer to a value of 680 however, we see the updated value. The interaction with the slicer sends a new query to the data source returning the updated results displayed in the filtered table.

Clearing the slicer shows the initial table again, without the updated value. Refreshing the report clears all caches and runs all queries required by the visuals on the page.


  • Power BI Desktop reports must be refreshed to reflect schema changes. Once you publish a report to the Power BI Service selecting Refresh only refreshes the visuals in the report. If the underlying schema changes Power BI will not automatically update the available field lists. Updating the data schema requires opening the .pbix file in Power BI Desktop, refresh the report, then republish the report.

Example
Below is an example the limitation noted above. Here, we have the same Power BI report on the right as the example above and the SQL database on the left. We will start by executing the query which adds a new ManagerID column to the Product table, sets the value as a random number between 0 and 100, and then selects the top 100 products to view the update.

After executing the query we can refresh the columns of the Product table in SQL Server Management Studio (SSMS) to verify it was created. However, in Power BI we see that the fields available to add to the table visual does not include this new column.

To view schema updates in Power BI the .pbix file must be refreshed and the report must be republished.

As noted above if columns or tables are removed from the underlying data source Power BI queries can break. To see this we first remove the ManagerId column in the data source, and then refresh the report. After refreshing the report we can see there is an issue with the table visual.


  • The limit of rows returned by a query is capped at 1 million rows.
  • The Power BI data model cannot be switched from Import to DirectQuery mode. A DirectQuery table can be switched to Import, however once this is done it cannot be switched back.
  • Some Power Query (M Language) features are not supported by DirectQuery. Adding unsupported features to the Power Query Editor will result in a This step results in a query that is not supported by DirectQuery mode error message.
  • Some DAX functions are not supported by DirectQuery. If used results in a Function <function_name> is not supported by DirectQuery mode error message.

Example
For the example report we only need the ProductIDName, and ListPrice field. However, you can see in the data pane we have all the columns present in the source data. We can modify which columns are available in Power BI be editing the query in the Power Query Editor.

After removing the unneeded columns we can view the native query that get executed against the underlying data source and see the SELECT statement includes only the required columns (i.e. column not removed by the Removed Columns step).


Other implications and considerations of DirectQuery include performance and load implications on the data source, data security implications, data-model limitations, and reporting limitations.

DirectQuery Use Cases

With its given limitations DirectQuery can still be a suitable option for the following use cases.

  • When report requirements include the need for near real-time data.
  • The report requires a large dataset, greater than what is supported by Import mode.
  • The underlying data source defines and applies security rules. When developing a Power BI report with DirectQuery Power BI connects to the data source by using the current user’s (report developer) credentials. DirectQuery allows a report viewer’s credentials to pass through to the underlying source, which then applies security rules.
  • Data sovereignty restrictions are applicable (e.g. data cannot be stored in the cloud). When using DirectQuery data is cached in the Power BI Service, however there is no long term cloud storage.

Live Connection

Live Connection is a method that lets you build a report in Power BI Desktop without having to build a dataset to under pin the report. The connectivity mode offloads as much work as possible to the underlying analysis services. When building a report in Power BI Desktop that uses Live Connection you connect to a dataset that already exists in an analysis service.

Similar to DirectQuery when Live Connection is used no data is imported or copied into the Power BI dataset. Rather Power BI stores a connection string to the existing analysis services (e.g. SSAS) or published Power BI dataset and Power BI is used as a visualization layer.

Live Connection Considerations

  • Can only be used with a limited number of data sources. Existing Analysis Service data model (SQL Server Analysis Services (SSAS) or Azure Analysis Services)
  • No data-model customizations are available, any changes required to the data-model need to be done at the data source. Report-Level measures are the one exception to this limitation.
  • User identity is passed through to the data source. A report is subject to row-level security and access permissions that are set on the data-model.

Composite Models

Power BI no longer limits you to choosing just Import or DirectQuery mode. With composite models the Power BI data-model can include data connections from one (or more) DirectQuery or Import data connections.

A composite model allows you to connect to different types of data sources when creating the Power BI data-model. Within a single .pbix file you can combine data from one or more DirectQuery sources and/or combine data from DirectQuery sources and Import data.

Each table within the composite model will list its storage mode which shows whether the table is based on a DirectQuery or Import source. The storage mode can be viewed and modified on the properties pane of the table.


Example
The example below is a Power BI report with a table visual of a products table. We have added the ManagerID column to this table, however there is no Managers table in the underlying SQL database. Rather this information is contained within a local product_managers Excel file. With a composite model we can combine these two different data sources and connectivity modes to create a single report.

Initially the report storage mode is DirectQuery because to start we only have the Product table DirectQuery connection.

We use the Import connectivity mode to load the product_mangers data and create the relationship between the two tables.

You can check the storage mode of each table in the properties pane under Advanced. We can see the SalesLT Product table has a DirectQuery storage mode and that Sheet1 has a Import storage mode.

Once the data model is a composite model we see the report Storage Mode update to a value of Mixed.


Composite Model Considerations

  • Security Implications: A query sent to one data source could include data values that have been extracted from a different data source. If the extracted data is confidential the security impacts of this should be considered. You should avoid extracting data from one data source via an encrypted connection to then include this data in a query sent to a different source via an unencrypted connection.
  • Performance Implications: Whenever DirectQuery is used the performance of the underlying system should be considered. Ensure that it has the resources required to support the query load due to users interacting with the Power BI report. A visual in a composite model can send queries to multiple data source with the results from one query being passed to another query from a different source

Thank you for reading! Stay curious, and until next time, happy learning.

And, remember, as Albert Einstein once said, “Anyone who has never made a mistake has never tried anything new.” So, don’t be afraid of making mistakes, practice makes perfect. Continuously experiment and explore new DAX functions, and challenge yourself with real-world data scenarios.

If this sparked your curiosity, keep that spark alive and check back frequently. Better yet, be sure not to miss a post by subscribing! With each new post comes an opportunity to learn something new.