Page History

...

Notes:

• Announcements:
  • We recently removed support for Airflow 1.x
  • Ross gave a talk on OpenLineage at ApacheCon in New Orleans last week
  • Upcoming opportunities to give talks about OpenLineage: 
  • Data Teams Summit (January 2023)
  • Subsurface Live (January 2023)
  • Data Council Austin  (March 2023)
  • Giving a talk on data lineage soon? Ping Michael R. on Slack to let us know.
• Recent release 0.15.11 [Michael R.]

  • Added

    • Airflow: improve development experience #1101 @JDarDagran
    • Documentation: update issue templates for proposal & add new integration template #1116 @rossturk
    • Spark: add description for URL parameters in readme, change overwriteName to appName #1130 @tnazarew

    Changed

    • Airflow: lazy load BigQuery client #1119 @mobuchowski

    Fixed

    • Spark: fix column lineage #1069 @pawel-big-lebowski
    • Spark: set log level of Init OpenLineageContext to DEBUG #1064 **new contributor @varuntestaz**
    • Java client: update version of SnakeYAML #1090 **new contributor Lukáš AKA @TheSpeedding**
    • CI: build macos release package on medium resource class #1131 @mobuchowski
    Additional bug fixes and more details: https://github.com/OpenLineage/OpenLineage/blob/main/CHANGELOG.md
• Project roadmap reviewreview [Harel]
  • Improved understanding of Airflow
  • Track DAG runs
  • Native lineage in operators
  • Increased adoption of OpenLineage consumers
  • Collaborate with data catalogs
  • Coverage by event producers
  • Increased support for Snowflake access history
  • Data quality frameworks
  • Start thinking about data consumption integrations
  • Continue experimenting with a Flink integration
  • Increased support of column level lineage
• Column-level lineage workshop [Howard]

September 8, 2022 (10am PT)

...