Skip to main content

1.3.1 - 2023-10-03

Added

  • Airflow: add some basic stats to the Airflow integration #1845 @harels
    Uses the statsd component that already exists in the Airflow codebase and wraps the section that emits to event with a timer, as well as emitting a counter for exceptions in sending the event.
  • Airflow: add columns as schema facet for airflow.lineage.Table (if defined) #2138 @erikalfthan
    Adds columns (if set) from airflow.lineage.Table inlets/outlets to the OpenLineage Dataset.
  • DBT: add SQLSERVER to supported dbt profile types #2136 @erikalfthan
    Adds support for dbt-sqlserver, solving #2129.
  • Spark: support for latest 3.5 #2118 @pawel-big-lebowski
    Integration tests are now run on Spark 3.5. Also upgrades 3.3 branch to 3.3.3. Please note that delta and iceberg are not supported for Spark 3.5 at this time.

Fixed

  • Airflow: fix find-links path in tox #2139 @JDarDagran
    Fixes a broken link.
  • Airflow: add more graceful logging when no OpenLineage provider installed #2141 @JDarDagran
    Recognizes a failed import of airflow.providers.openlineage and adds more graceful logging to fix a corner case.
  • Spark: fix bug in PathUtils' prepareDatasetIdentifierFromDefaultTablePath(CatalogTable) to correctly preserve scheme from CatalogTable's location #2142 @d-m-h
    Previously, the prepareDatasetIdentifierFromDefaultTablePath method would override the scheme with the value of "file" when constructing a dataset identifier. It now uses the scheme of the CatalogTable's URI for this. Thank you @pawel-big-lebowski for the quick triage and suggested fix.