Discover how to use Apache Spark and Python for data ingestion into a Microsoft Fabric lakehouse. Fabric notebooks provide a scalable and systematic solution. In this lab, you’ll create a Microsoft Fabric notebook and use PySpark to connect to an Azure Blob Storage path, then load the data into a lakehouse using write optimizations. For this experience, you’ll build the code across multiple notebook code cells, which may not reflect how you will do it in your environment; however, it can be useful for debugging. Because you’re also working with a sample dataset, the optimization doesn’t reflect what you may see in production at scale; however, you can still see improvement and when every millisecond counts, optimization is key. The video is based on Microsoft Lab which supports "Ingest data with Spark and Microsoft Fabric notebooks" module in Microsoft Learn (see links below). What you'll find in this video: 00:00 Intro 00:57 Lakehouse destination 02:43 Create notebook and load external data 10:54 Transform and load data to a Delta table 17:48 Analyze Delta table data with SQL queries 20:06 Summary *** Useful links: *** Whole learning path: Module: Lab10: *** Socials: *** - 📓 GitHub: - 📢 Twitter: - 📢 Twitter: - 👥 Facebook: - 🏙️ LinkedIn: - 🖼️ Instagram: - ⭐️ Microsoft MVP Profile: *** Hungry more? Learn with me *** - 👨‍🎓 Courses: - 🌏 Blog: - ✉️ Newsletter: If you're reading this, please leave a like and comment for the algorithm.