Power Query
Cookbook
Use effective and powerful queries in Power BI Desktop and Dataflows to prepare and transform your data
Andrea Janicijevic
BIRMINGHAM—MUMBAI
Power Query Cookbook
Copyright © 2021 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Group Product Manager: Kunal Parikh
Publishing Product Manager: Ali Abidi
Senior Editor: Roshan Kumar
Content Development Editor: Tazeen Shaikh
Technical Editor: Rahul Limbachiya
Copy Editor: Safis Editing
Project Coordinator: Aparna Ravikumar Nair
Proofreader: Safis Editing
Indexer: Subalakshmi Govindhan
Production Designer: Prashant Ghare
First published: October 2021
Production reference: 1070921
Published by Packt Publishing Ltd.
Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-80056-948-5
www.packt.com
To my family, who have always supported me during my studies and my professional experience and have been key in me becoming the woman I am today. To my colleague, Patrik Borosch, who introduced me to this opportunity, and to my manager, Zoran Draganic, who gave me the freedom to pursue this exciting journey.
– Andrea Janicijevic
Contributors
About the author
Andrea Janicijevic is a cloud solution architect and works in the world of analytics and business intelligence, constantly expanding her knowledge in the field of data. From the outset, she has been working on analytics platforms, helping clients to better adopt cloud technology across a wide range of industries and company sizes.
She studied economics and management of innovation and technology at Bocconi University in Milan and during her studies, she started working at Microsoft in 2018. She began working with the Microsoft analytics platform, including Power BI, becoming a trusted technical advisor for business and technical users. She later started collaborating with Packt, accepting the challenge of sharing her experience with Power Query.
About the reviewers
Patrik Borosch is a cloud solution architect for data and AI at Microsoft Switzerland GmbH. He has more than 25 years of BI and analytics development, engineering, and architecture experience and is a Microsoft Certified Data Engineer and a Microsoft Certified AI Engineer. Patrik has worked on numerous significant international data warehouse, data integration, and big data projects. Through this, he has built and extended his experience in all facets, from requirements engineering to data modeling and ETL, all the way to reporting and dashboarding. At Microsoft Switzerland, he supports customers in their journey into the analytical world of the Azure Cloud.
Michiel Rozema is one of Europe's top Power BI experts, living in the Netherlands. He holds a master's degree in mathematics and has worked in the IT industry for over 25 years as a consultant and manager. Michiel was the data insight lead at Microsoft
Netherlands for 8 years, during which time he launched Power BI in the country. He is the author of two Dutch books on Power Pivot and Power BI, and is the author of the Extreme DAX title with Packt Publishing. Michiel is one of the founders of the Dutch Power BI user group and the initiator of the Power BI Summer School, and has been a speaker at many conferences on Power BI. He has been awarded the Microsoft MVP award since 2019 and, together with fellow MVP Henk Vlootman, runs the consultancy firm Quanto, specializing in Power BI.
Table of Contents
Preface
1
Getting Started with Power Query
Technical requirements |
2 |
Installing a Power BI gateway |
3 |
Getting ready |
3 |
How to do it… |
4 |
How it works |
15 |
Authentication to data sources |
15 |
Getting ready |
16 |
How to do it… |
16 |
How it works |
25 |
Main challenges that Power |
|
Query solves |
25 |
Getting ready |
25 |
How to do it… |
25 |
2
Connecting to Fetch Data
Technical requirements
Getting data and connector navigation
Getting ready How to do it...
Creating a query from files
Getting ready How to do it...
How it works...
Creating a query from a folder
Getting ready
36 |
How to do it... |
50 |
|
How it works... |
55 |
36 |
Creating a query from a |
|
36 |
|
|
database |
56 |
|
36 |
Getting ready |
56 |
|
||
39 |
How to do it... |
56 |
40 |
How it works... |
62 |
|
|
|
41 |
Creating a query from a website |
62 |
49 |
Getting ready |
63 |
49 |
How to do it... |
63 |
49 |
How it works... |
69 |
viii Table of Contents
3
Data Exploration in Power Query
Technical requirements |
72 |
Using Queries pane shortcuts |
91 |
Exploring Power Query Editor |
72 |
Getting ready |
91 |
Getting ready |
73 |
How to do it… |
91 |
How to do it… |
73 |
Using Query Settings pane |
|
|
|
|
|
Managing columns |
78 |
shortcuts |
93 |
Getting ready |
78 |
Getting ready |
93 |
How to do it… |
78 |
How to do it… |
93 |
Using data profiling tools |
82 |
Using Schema view and |
|
Getting ready |
82 |
Diagram view |
96 |
How to do it… |
82 |
Getting ready |
96 |
|
|
How to do it… |
97 |
4
Reshaping Your Data
Technical requirements |
108 |
Getting ready |
129 |
Formatting data types |
108 |
How to do it |
130 |
Getting ready |
108 |
Splitting columns |
132 |
How to do it |
109 |
Getting ready |
132 |
Using first rows as headers |
118 |
How to do it |
132 |
Getting ready |
118 |
Extracting data |
136 |
How to do it |
118 |
Getting ready |
137 |
|
|
||
Grouping data |
120 |
How to do it |
137 |
Getting ready |
120 |
Parsing JSON or XML |
139 |
How to do it |
121 |
Getting ready |
139 |
Unpivoting and pivoting |
|
How to do it |
140 |
columns |
124 |
Exploring artificial intelligence |
|
Getting ready |
124 |
insights |
143 |
How to do it |
124 |
Getting ready |
143 |
|
|
||
Filling empty rows |
129 |
How to do it |
144 |
Table of Contents ix
5
Combining Queries for Efficiency
Technical requirements |
150 |
How to do it… |
165 |
|
Merging queries |
150 |
Combining multiple files |
169 |
|
Getting ready |
150 |
Getting ready |
170 |
|
How to do it… |
151 |
|||
How to do it… |
170 |
|||
|
|
|||
Joining methods |
156 |
Using the Query Dependencies |
|
|
Getting ready |
156 |
|
||
view |
173 |
|||
How to do it… |
156 |
Getting ready |
173 |
|
|
|
|||
Appending queries |
165 |
How to do it… |
174 |
|
Getting ready |
165 |
|
|
6
Optimizing Power Query Performance
Technical requirements |
182 |
How to do it… |
204 |
Setting up parameters |
182 |
Leveraging incremental refresh |
|
Getting ready |
182 |
|
|
and folding |
212 |
||
How to do it… |
183 |
Getting ready |
213 |
|
|
||
Filtering with parameters |
189 |
How to do it… |
213 |
Getting ready |
189 |
Disabling query load |
223 |
How to do it… |
189 |
Getting ready |
223 |
Folding queries |
203 |
How to do it… |
224 |
Getting ready |
204 |
|
|
7
Leveraging the M Language
Technical requirements |
232 |
Using M and DAX – differences |
238 |
|
Using M syntax and the |
|
Getting ready |
238 |
|
Advanced Editor |
232 |
How to do it… |
238 |
|
Getting ready |
232 |
Using M on existing queries |
243 |
|
How to do it… |
233 |
|||
|
|
x Table of Contents
Getting ready |
243 |
Creating tables in M |
259 |
How to do it… |
243 |
Getting ready |
259 |
Writing queries with M |
250 |
How to do it… |
260 |
|
|
||
Getting ready |
250 |
Leveraging M – tips and tricks |
264 |
How to do it… |
251 |
Getting ready |
264 |
|
|
How to do it… |
264 |
8
Adding Value to Your Data
Technical requirements |
270 |
Getting ready |
283 |
Adding columns from examples 270 |
How to do it… |
284 |
|
Getting ready |
270 |
Invoking custom functions |
289 |
How to do it… |
270 |
Getting ready |
289 |
Adding conditional columns |
278 |
How to do it… |
290 |
Getting ready |
278 |
Clustering values |
298 |
How to do it… |
279 |
Getting ready |
298 |
Adding custom columns |
283 |
How to do it… |
298 |
9
Performance Tuning with Power BI Dataflows
Technical requirements |
304 |
Building dataflows with Power |
|
|
Using Power BI dataflows |
304 |
BI Premium capabilities |
326 |
|
Getting ready |
304 |
Getting ready |
326 |
|
How to do it... |
305 |
How to do it... |
327 |
|
Centralizing ETL with dataflows 314 |
Understanding dataflow best |
341 |
||
Getting ready |
315 |
practices |
||
Getting ready |
342 |
|||
How to do it... |
315 |
|||
How to do it... |
342 |
|||
|
|
|||
Table of Contents xi
10
Implementing Query Diagnostics
Technical requirements |
353 |
Designing a report with |
|
|
Exploring diagnostics options |
354 |
diagnostics results |
363 |
|
Getting ready |
354 |
Getting ready |
363 |
|
How to do it… |
354 |
How to do it… |
364 |
|
Managing a diagnostics session 358 |
There's more… |
379 |
||
Getting ready |
358 |
Using Diagnose as a Power |
|
|
Query step |
379 |
|||
How to do it… |
358 |
|||
|
|
Getting ready |
379 |
|
|
|
How to do it… |
380 |
|
Other Books You May Enjoy
Index