Articles by Tag #etl
Browse our collection of articles on various topics related to IT technologies. Dive in and explore something new!
Using Pydantic for ETL - Clean, Validate, and Transform Data with Confidence
In most data workflows, the ETL process - Extract, Transform, Load - is where everything begins....
From PDFs to Markdown
Introduction Retrieval-Augmented Generation (RAG) pipelines rely heavily on accurate and...
Bicep / ARM vs Terraform
Bicep and ARM templates are Terraform alternatives, but only within the Azure ecosystem. ...
Cómo solucioné el error “The property 'ParameterName' contains invalid characters” en SSIS
Introducción Mientras trabajaba con proyectos de Integration Services (SSIS) en SQL Server...
The 5 Most Common Data Quality Issues (and How Analysts Can Fix Them)
Data analysts spend more time cleaning data than analyzing it. In fact, in most real-world projects,...
Python For Data Engineering
Data engineers are responsible for managing, processing, and transforming raw data into valuable...
ETL Made Easy: Integrating Multi-Source Data with AWS Glue
AWS Glue is a serverless data integration service that you can use to perform Extract, Transform, and...
Popular tools used in ETL/ELT workflows
I compiled this categorized list using chatgpt, of popular tools used in ETL/ELT workflows ...
How to Data Engineer the ETLFunnel Way
Part 1 — Idempotency, Retry, and Recovery Modern data engineering isn't about moving data...
Which is Best for Real Time Dashboards: Airbyte, Fivetran, or Estuary
A dashboard is only as valuable as the freshness of the data behind it. If the numbers are hours old,...
🔄 ETL vs ELT: The Backbone of Data Engineering
In the world of Data Engineering, two terms come up all the time: ETL and ELT. While they sound...
“𝗘𝗧𝗟 𝗶𝘀 𝗘𝘃𝗼𝗹𝘃𝗶𝗻𝗴 — 𝗔𝗿𝗲 𝗬𝗼𝘂?”
The hidden reason your data pipeline feels slow. ETL isn’t old-school. It’s becoming smarter in...
Unlock the Power of LLM-Driven ETL: Transform Variable CSV to Clean JSON with C#, Semantic Kernel & Llama 3.2-3B
Introduction We'll feed a messy CSV file to a lightweight llama3.2-3B model and let it...
Well-formed, Valid, Canonical, and Correct
The world of data is messy in just about every dimension. One of those dimensions is how we talk...
🌿 Building a Resilient NDVI Data Pipeline: From Google Earth Engine to NetCDF with Airflow on WSL
An in-depth technical dive into automating satellite data workflows, tackling real-world issues, and...
RDS MySQL Zero ETL Integration with Redshift
Introduction: Data integration is one of the most critical aspects of any data-driven...
Mastering SQL for Data Engineering: Advanced Queries, Optimization, and Data Modeling Best Practices
Advanced SQL for Data Engineering Querying a database should be at your fingertips. This...
Create a cross-account glue Job using AWS CDK
AWS Glue is a powerful service for data integration and ETL (Extract, Transform, Load) workloads,...
How JavaScript helped me find a Car
I'm looking for a Jetta, and of course, I want to make sure I get the best option. Instead of...
ETL e ELT
ETL Extração, transformação e carregamento (ETL) correspondem ao processo de combinação de...
Understanding Data Pipelines: The Backbone of Modern Data Systems
In today’s data-driven world, organizations are collecting vast amounts of data from various...
Writing Maintainable ETL Code: Empowering Support and Avoiding Knowledge Silo
Writing Maintainable ETL Code: Think Beyond Just the Developer When we write ETL code, our...
Top ETL Tools for MongoDB in 2025: Which One Fits Your Use Case?
This article was written by Radhika Sarraf. Modern businesses generate data from countless sources:...
Stop Writing Nested Loops for ETL. Compile Them with convtools
convtools is a tiny Python library (GitHub link) that turns declarative transforms into plain...
Building an ETL Pipeline with Python Using CoinGecko API
Extract, Transform, Load (ETL) is a fundamental process in data engineering used to collect data from...
Maximizing Business Growth: The Power of HubSpot and Dynamics 365 Integration
The article was initially published on the Skyvia blog. When it comes to integrating HubSpot and...
Are LLMs Just ETL Pipelines on Steroids? Rethinking AI Training
We talk a lot about Large Language Models (LLMs) like GPT, Claude, and Llama – their incredible...
Implementing Real-Time Data Processing Using Apache Flink
In today’s fast-paced digital landscape, the ability to process data in real-time is invaluable for...
InsightFlow Part 6: Implementing ETL Processes with AWS Glue for InsightFlow
InsightFlow GitHub Repo In this post, we’ll explore how AWS Glue was used to implement the ETL...
How Automation Can Simplify Your Sales and Accounting Workflow
This article was initially published on the Skyvia blog. For businesses looking to optimize their...