Skip to main content

1.6.2 - 2023-12-07

Added

  • Dagster: support Dagster 1.5.x #2220 @tsungchih
    Gets event records for each target Dagster event type to support Dagster version 0.15.0+.
  • Dbt: add a new command dbt-ol send-events to send metadata of the last run without running the job #2285 @sophiely
    Adds a new command to send events to OpenLineage according to the latest metadata generated without running any dbt command.
  • Flink: add option for Flink job listener to read from Flink conf #2229 @ensctom
    Adds option for the Flink job listener to read jobnames and namespaces from Flink conf.
  • Spark: get column-level lineage from JDBC dbtable option #2284 @mobuchowski
    Adds support for dbtable, enables lineage in the case of single input columns, and improves dataset naming.
  • Spec: introduce JobTypeJobFacet to contain additional job related information#2241 @pawel-big-lebowski
    New JobTypeJobFacet contains the processing type such as BATCH|STREAMING, integration via SPARK|FLINK|... and job type in QUERY|COMMAND|DAG|....
  • SQL: add quote information from sqlparser-rs #2259 @JDarDagran
    Adds quote information from sqlparser-rs.

Fixed

  • Spark: update Jackson dependency to resolve CVE-2022-1471 #2185 @pawel-big-lebowski
    Updates Gradle for Spark and Flink to 8.1.1. Upgrade Jackson 2.15.3.
  • Flink: avoid relying on Guava which can be missing during production runtime #2296 @pawel-big-lebowski
    Removes usage of Guava ImmutableList.
  • Spark: exclude commons-logging transitive dependency from published jar #2297 @pawel-big-lebowski
    Ensures commons-logging is not shipped as this can lead to a version mismatch on the user's side.