Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.


  • Announcements [Julien]
    • Two meetups will be happening soon:
      • Data Lineage Meetup cohosted with Collibra, Providence, RI, March 9 at 6 PM ET
      • OpenLineage Meetup at Data Council Austin on March 30th at 12:15 PM CST
    • Talk happening soon:
      • Julien Le Dem, "Ten Years of Building Open Source Standards: From Parquet to Arrow to OpenLineage," Data Council Austin, March 30th, 10 AM CST
  • Recent releases 0.20.6, 0.21.1
  • Custom env var support in the Spark integration [Anirudh]
    • adds ability to capture environment variables from a Spark cluster
    • required the addition of a new class to extend an existing class
    • does not override variables already being captured
    • desired variables must be specified by the user
    • variables are visible in environment properties of OpenLineage events
    • Q & A
      • Q: is it possible to accidentally include sensitive data in these variables?
      • A: users must "opt in" by selecting variables in advance
      • Q: what was the experience like interacting with the community?
      • A: really great! I got a lot of help from a lot of people, including Pawel

February 9, 2023 (10am PT)