Page History

...

• TSC:
  • Julien Le Dem, OpenLineage project lead
  • Maciej Obuchowski, Software Engineer, GetInData, OpenLineage committer
  • Michael Robinson, Community team, Astronomer
• And:
  • Cori Visi, Solutions Architect, AWS
  • Harel Shein, Engineering Manager, Astronomer
  • John Lukenoff, Software Engineer, Asana
  • Suparna Bhattacharya, HPE Labs
  • Ann Mary Justine, Research Engineer, HP Enterprise's CMF team
  • Anirudh Shrinivason, Data Engineer, Grab
  • Chris Olivares, CTO at , Hum Capital 
  • Martin Foltin, HPE Research Labs
  • Sheeri Cabral, Technical Product Manager, Lineage, Collibra
  • Harry, works at a Bay area-based fintech firm
  • Julian LaNeve, Senior Product Manager, Astronomer

...

1. Announcements
2. Recent releases
3. Static lineage progress update
4. Open discussion

Meeting:

Notes:

1. Announcements [Julien]:
   1. Our first annual ecosystem survey is live and accepting responses: https://bit.ly/ecosystem_survey. Your participation matters!
   2. We recently published the first issue of our monthly newsletter: https://mailchi.mp/18826f97904e/openlineage-news-may-2023. It's a great way to learn about upcoming meetups and recent blog posts, etc.
   3. Two meetups are happening soon:
      1. New York on 6/22 at Collibra's HQ: https://www.meetup.com/data-lineage-meetup/events/294065396/
      2. San Francisco on 6/27 at Astronomer: https://www.meetup.com/meetup-group-bnfqymxe/events/293448130/
   4. Upcoming talks:
      1. Paweł Leszczyński and Maciej Obuchowski, “Column Lineage is Coming to the Rescue,” Berlin Buzzwords, June 18-20, 2023
      2. Julien Le Dem and Willy Lulciuc, “Cross-platform Data Lineage with OpenLineage,” Data+AI Summit, June 28-29, 2023
      3. Maciej Obuchowski, “OpenLineage in Airflow: A Comprehensive Guide,” Airflow Summit, September 19-21, 2023
2. Recent releases [Michael R.]:
   1. OpenLineage 0.25.0
      • Added
        • Spark: add Spark/Delta merge into support #1823 @pawel-big-lebowski
      • https://github.com/OpenLineage/OpenLineage/releases/tag/0.25.0
      • https://github.com/OpenLineage/OpenLineage/compare/0.24.0...0.25.0
   2. OpenLineage 0.26.0
      • Added
        • Proxy: Fluentd proxy support (experimental) #1757 @pawel-big-lebowski
      • Changed
        • Python client: use Hatchling over setuptools to orchestrate Python env setup #1856 @gaborbernat
      • https://github.com/OpenLineage/OpenLineage/releases/tag/0.26.0
      • https://github.com/OpenLineage/OpenLineage/compare/0.25.0...0.26.0
   3. OpenLineage 0.27.1
      • Added
        • Python client: add emission filtering mechanism and exact, regex filters #1878 @mobuchowski
      • https://github.com/OpenLineage/OpenLineage/releases/tag/0.27.1
      • https://github.com/OpenLineage/OpenLineage/compare/0.26.0...0.27.1
   4. OpenLineage 0.27.2
      • Fixed
        • Python client: deprecate client.from_environment, do not skip loading config #1908 @mobuchowski
      • https://github.com/OpenLineage/OpenLineage/releases/tag/0.27.2
      • https://github.com/OpenLineage/OpenLineage/compare/0.27.1...0.27.2

May 11, 2023 (10am PT)

Attendees:

...