Войти
  • 258471Просмотров
  • 3 года назадОпубликованоBI Insights Inc

How to build an ETL pipeline with Python | Data pipeline | Export from SQL Server to PostgreSQL

In this video, we will discuss what ETL is. ETL stands for Extract, Transform, Load. ETL is a set of processes that extracts data from one or more sources (API, a database or a file), transforms it to match the destination system’s requirements and loads it into the destination system. In this tutorial we will see how to design ETL Pipeline with Python. We will use SQL Server’s AdventureWorks database as a source and load data in PostgreSQL with Python. We often have requirements for data analytics and/or reporting projects that requires optimized data for querying. ETL moves and amalgamate the data from various sources and stores in the destination where it is available for data analytics, reporting. Furthermore, data scientist and data analysts can use this optimized data come up with new findings. Link to medium article on this topic: @hnawaz100/how-to-build-an-etl-pipeline-with-python-1b78407c3875 Link to Microsoft ODBC driver: Link to ETL Script: %20Pipeline/ Link to GitHub repo: %20Pipeline Link to the Python ETL series: How to connect to SQL Server via Python: SQL Server install video: PostgreSQL Install video: Subscribe to our channel: --------------------------------------------- Follow me on social media! Github: Instagram: LinkedIn: --------------------------------------------- #ETL #Python #SQL Topics covered in this video: 0:00 - Introduction to ETL 1:55 - PostgreSQL Setup 2:49 - Sqlserver Setup 3:32 - Code ETL Pipeline 5:10 - Data Extract from SQL Server 7:30 - Load data to PostgreSQL 9:22 - Test ETL Pipeline