dlt
A Python library for declarative data loading (data load tool) that simplifies building data pipelines to extract from APIs and load into S3-based data lakes and lakehouses with automatic schema inference and evolution handling.
Summary
A Python library for declarative data loading (data load tool) that simplifies building data pipelines to extract from APIs and load into S3-based data lakes and lakehouses with automatic schema inference and evolution handling.
dlt is a lightweight, code-first alternative to heavier orchestration tools for getting data into S3. It targets Python-centric data teams who want pipeline-as-code without managing Airbyte infrastructure or writing custom Spark jobs.
- dlt is a Python library, not a managed service. It runs wherever Python runs (local, Airflow, Lambda) but requires the user to handle scheduling, monitoring, and failure recovery.
- Schema inference is automatic but not infallible. Unexpected source data types or nullable fields can cause schema evolution that downstream consumers are not prepared for.
- dlt's S3 destination writes files but does not manage table format metadata. For Iceberg/Delta integration, dlt relies on destination-specific adapters.
scoped_toS3, Lakehouse — loads data into S3-based destinationsalternative_toAirbyte — lightweight code-first alternativeenablesEvent-Driven Ingestion — pipeline-as-code for event-triggered loadsconstrained_bySchema Evolution — automatic schema changes can propagate unexpectedly
Definition
An open-source Python library for declarative data loading that extracts data from APIs, databases, and files, and loads it into S3-based destinations including data lakes and lakehouses with automatic schema evolution.
Traditional ETL frameworks require extensive boilerplate for schema management, incremental loading, and error handling. dlt provides a Python-native, declarative approach to data loading that handles schema inference and evolution automatically when writing to S3.
Python-native ELT pipelines to S3, automated schema evolution during ingestion, lightweight data loading without orchestration overhead.
Recent developments
MCP-server integration brings AI-assisted pipeline authoring. Per the dlt-hub organization repository activity (updated April 26, 2026), dlt now ships an MCP server that integrates with VS Code Copilot for AI-assisted pipeline development. A reference build is documented in stephandoh/zoomcamp_DE_DLT_2026 — building an NYC taxi-data ingestion pipeline using the dlt MCP server in VS Code Copilot, demonstrating the AI-pair-programming flow with dlt + DuckDB. This is the same architectural pattern Snowflake and Databricks are pushing for their MCP integrations: pipeline authoring becomes a conversation with the catalog rather than schema-by-schema YAML.
Repo footprint stable: 5.3k stars, 499 forks, broad schema-inference + incremental-loading coverage. Per the dlt-hub/dlt repository, the core library covers schema inference, incremental loading, and normalization across the same destinations that Fivetran-class tools serve — but with a Python-library shape rather than a managed SaaS shape. The library remains the recommended Python-first ELT path for teams that want pipelines as code rather than pipelines as config.
dlt 1.27 shipped native Polars DataFrame support and Databricks Zerobus loading; Python 3.9 support dropped. The March 2026 release line (1.27.0, hotfixed to 1.27.2) also added incremental filtering for
dlt.Relationalongside a documentation overhaul. Per dlt-hub/dlt Releases.dltHub's Arrow + ADBC pipeline path runs 3.7x faster than SQLAlchemy for extract-and-load jobs. In dltHub's own benchmark, loading 5M rows from DuckDB to MySQL dropped from 344s to 92s, and normalization time fell from 167s to roughly 10ms. Per 3.7x Faster EL Pipelines: Arrow + ADBC vs. SQLAlchemy.
dlt reports 3 million monthly downloads and 5,000+ companies in production, covering 4,600+ REST API sources. These adoption figures place it as one of the most widely deployed Python-native ELT libraries, alongside its MCP-server pipeline-authoring work. Per VentureBeat — How dltHub's Python open-source platform transforms data pipeline creation. Sources: dlt-hub/dlt (GitHub) · dlt-hub organization repositories · zoomcamp_DE_DLT_2026 — dlt MCP server reference build
Connections7
Outbound6
depends_on1alternative_to1Inbound1
alternative_to1Resources3
Official dlt (data load tool) documentation for the Python library that simplifies building data pipelines with automatic schema inference and S3 destinations.
dlt source repository with the pipeline framework, filesystem destination, and Parquet/Iceberg integration code.
dlt filesystem destination guide covering S3 writes with partitioning, Parquet output, and Delta/Iceberg table format support.