Page History

...

All are welcome.

...

March 9, 2023 (10am PT)

Tentative agenda:

• Recent releases
• APIDemo: a new OpenLineage in Airflow
• Discussion topic: real-world implementation of OpenLineage (i.e., "What IS lineage, anyway?")
• consumer
• Discussion Announcement & discussion topic: the thinking behind namespaces
• Open discussion

Meeting:

...

February 9, 2023 (10am PT)

Attendees:

• TSC:
  • Julien Le Dem, OpenLineage project lead
  • Ross Turk, Senior Director of Community, Astronomer
  • Benji Lampel, Product Manager, Astronomer
  • Minkyu Park, Senior Engineer, Astronomer
  • Michael Collado, Staff Engineer, Astronomer
  • Howard Yoo, Staff Product Manager, Astronomer
  • Maciej Obuchowski, Software Engineer, GetInData, OpenLineage contributor
  • Willy Lulciuc, Co-creator of Marquez
  • Danny Henneberger, OpenLineage committer
  • Michael Robinson, Developer Relations Engineer, Astronomer
• And:
  • Prachi Mishra, Senior Software Engineer, Astronomer
  • Sheeri Cabral, Project Manager, Collibra
  • Enrico Rotundo, Bacalhau Project
  • Brad, Fivetran
  • Harel Shein, Director of Engineering, Astronomer
  • Robert Karish, Data Engineer, AdTheorent
  • Eric Veleker, Atlan
  • Ben Sandler
  • Peter Hicks, Senior Software Engineer, Astronomer
  • John Thomas, Software Engineer, Developer Relations, Astronomer
  • Nikhil Wadhwa, Engineer, Fivetran
  • Sam Holmberg, Senior Software Engineer, Astronomer
  • David
  • Matthew Krubski

Agenda:

• Recent releases
• AIP: OpenLineage in Airflow
• Discussion topic: real-world implementation of OpenLineage (i.e., "What IS lineage, anyway?")
• Announcement & discussion topic: the thinking behind namespaces
• Open discussion

Meeting:

Notes:

• Announcements [Julien]
  • The first Data Lineage Meetup will be taking place in Providence on March 9th at 6 pm. More information: https://openlineage.io/blog/data-lineage-meetup/
• Recent release 0.20.4 [Michael R.]

  • Added

    • Airflow: add new extractor for GCSToGCSOperator#1495@sekikn
      Adds a new extractor for this operator.
    • Flink: resolve topic names from regex, support 1.16.0 #1522@pawel-big-lebowski
      Adds support for Flink 1.16.0 and makes the integration resolve topic names from Kafka topic patterns.
    • Proxy: implement lineage event validator for client proxy #1469@fm100
      Implements logic in the proxy (which is still in development) for validating and handling lineage events.

    Changed

    • CI: use ruff instead of flake8, isort, etc., for linting and formatting #1526@mobuchowski
      Adopts the ruff package, which combines several linters and formatters into one fast binary.
  • Thanks to all our contributors!
  • More details: https://github.com/OpenLineage/OpenLineage/blob/main/CHANGELOG.md

January 12, 2023 (10am PT)

...