...
- Zoom recording
- Password: 1VJ=K5&X
Notes:
- Releases
- 0.8.2
Added
- openlineage-airflow now supports getting credentials from Airflows secrets backend (#723) @mobuchowski
- openlineage-spark now supports Azure Databricks Credential Passthrough (#595) @wjohnson
- openlineage-spark detects datasets wrapped by ExternalRDDs (#746) @collado-mike
Fixed
- PostgresOperator fails to retrieve host and conn during extraction (#705) @sekikn
- SQL parser accepts lists of sql statements (#734) @mobuchowski
- 0.8.1
Added
- Airflow integration uses new TaskInstance listener API for Airflow 2.3+ (#508) @mobuchowski
- Support for HiveTableRelation as input source in Spark integration (#683) @collado-mike
- Add HTTP and Kafka Client to openlineage-java lib (#480) @wslulciuc, @mobuchowski
- New SQL parser, used by Postgres, Snowflake, Great Expectations integrations (#644) @mobuchowski
Fixed
GreatExpectations: Fixed bug when invoking GreatExpectations using v3 API (#683) @collado-mike
- 0.7.1
Added
- Python implements Transport interface - HTTP and Kafka transports are available (#530) @mobuchowski
- Add UnknownOperatorAttributeRunFacet and support in lineage backend (#547) @collado-mike
- Support Spark 3.2.1 (#607) @pawel-big-lebowski
- Add StorageDatasetFacet to spec (#620) @pawel-big-lebowski
- README.md created at OpenLineage/integrations for compatibility matrix (#663) @howardyoo
Fixed
- Airflow: custom extractors lookup uses only get_operator_classnames method (#656) @mobuchowski
- Dagster: handle updated PipelineRun in OpenLineage sensor unit test (#624) @dominiquetipton
- Delta improvements (#626) @collado-mike
- Fix SqlDwDatabricksVisitor for Spark2 (#630) @wjohnson
- Airflow: remove redundant logging from GE import (#657) @mobuchowski
- Fix Shebang issue in Spark's wait-for-it.sh (#658) @mobuchowski
- Update parent_run_id to be a uuid from the dag name and run_id (#664) @collado-mike
- Spark: fix time zone inconsistency in testSerializeRunEvent (#681) @sekikn
- 0.8.2
Apr 13th, 2022 (9am PT)
Attendees:
...