Page History

...

• Recent releases (0.13.0, 0.13.1, 0.14.0, 0.14.1)
• Native data quality in Airflow with OpenLineage
• MANTA integrations using OpenLineage

Meeting:

Notes:

• Recent releases (0.13.0, 0.13.1, 0.14.0, 0.14.1) [Michael R.]
  • 0.13.0

    • Added

      • Add BigQuery check support #960 @denimalpaca
      • Add RUNNING EventType in spec and Python client #972 @mzareba382
      • Use databases & schemas in SQL Extractors #974 @JDarDagran
      • Implement Event forwarding feature via HTTP protocol #995 @howardyoo
      • Introduce SymlinksDatasetFacet to spec #936 @pawel-big-lebowski
      • Add Azure Cosmos Handler to Spark integration #983 @hmoazam
      • Support OL Datasets in manual lineage inputs/outputs #1015 @conorbev
      • Create ownership facets #996 @julienledem

      Changed

      • Use RUNNING EventType in Flink integration for currently running jobs #985 @mzareba382
      • Convert task object into JSON encodable when creating Airflow version facet #1018 @fm100

      Fixed

      • Add support for custom SQL queries in v3 Great Expectations API #1025 @collado-mike
  • 0.13.1

    • Fixed

      • Rename all parentRun occurrences to parent from Airflow integration #1037 @fm100
      • Do not change task instance during on_running event #1028 @JDarDagran
  • 0.14.0

    • Added

      • Support ABFSS and Hadoop Logical Relation in Column-level lineage #1008 @wjohnson
      • Add Kusto relation visitor #939 @hmoazam
      • Add ColumnLevelLineage facet doc #1020 @julienledem
      • Include symlinks dataset facet #935 @pawel-big-lebowski
      • Add support for dbt 1.3 beta's metadata changes #1051 @mobuchowski
      • Support Flink 1.15 #1009 @mzareba382
      • Add Redshift dialect to the SQL integration #1066 @mobuchowski

      Changed

      • Make the timeout configurable in the Spark integration #1050 @tnazarew

      Fixed

      • Add a dialect parameter to Great Expectations SQL parser calls #1049 @collado-mike
      • Fix Delta 2.1.0 with Spark 3.3.0 #1065 @pawel-big-lebowski
  • 0.14.1

    • Fixed

      • Fix Spark integration issues including error when no openlineage.timeout #1069 @pawel-big-lebowski
  • Notes:
    • Thank you to all the contributors! And a special shout out to new contributor Hanna Moazam!
• Native data quality in Airflow with OpenLineage
• MANTA integrations using OpenLineage

Meeting:

Notes:

• Recent releases (0.13.0, 0.13.1, 0.14.0, 0.14.1) [Michael R.]
• Native data quality in Airflow with OpenLineage [Benji]
  • Related webinar: https://www.astronomer.io/events/webinars/implementing-data-quality-checks-in-airflow/
• MANTA integrations using OpenLineage [Petr]

...