Page History

...

Design the connector

Map to the Egeria meta-model concepts

The best place to start with the design work is to understand the meta-model of Egeria itself. Consider how your metadata repository will map to the fundamental Egeria metadata concepts: Entities, Classifications, and Relationships.

...

For example, in the implementation of the sample IGC connector we suggest using categories with specific names in IGC to represent certain classifications. Additionally, one of the reasons for implementing a read-only connector is that we can still handle relationships without any properties: by simply having the properties of any Egeria relationships we translate from IGC all be empty.

Map to the Egeria open metadata types

Once you have some idea for how to handle the mapping to the meta-model concepts, check your thinking by working through a few examples. Pick a few of the open metadata types and work out on paper how they map to your metadata repository’s pre-existing model. Common areas to do this would be e.g. GlossaryTerm, GlossaryCategory for glossary (business vocabulary) content; RelationalColumn, etc for relational database structures; and so on.

...

• Ignore any open metadata types that do not map to your pre-existing types.
• Add any Egeria open metadata types that do not exist in your repository.
• Add Egeria open metadata properties to your pre-existing types when Egeria has additional properties that do not yet exist in your type(s).
• Implement a read-only connection (possibly with some hard-coding of property values) for types that are partially map-able, but not easily extended to support the full set of properties defined on the open metadata type.
• and so on.

Setup the connector project

...

Naturally change the version to whichever version of Egeria you'd like to build against. The dependencies listed ensure you'l have the necessary portion of Egeria to build your connector against.

Implement the repository connector

Within this adapter module implement the following:

Implement an OMRSRepositoryConnectorProvider

Start by writing an OMRSRepositoryConnectorProvider specific to your connector, which extends OMRSRepositoryConnectorProviderBase. The connector provider is a factory for its corresponding connector. Much of the logic needed is coded in the base class, and therefore your implementation really only involves defining the connector class and setting this in the constructor.

...

Note that you'll need to define a unique GUID for the connector type, and a meaningful name and description. Really all you then need to implement is the constructor, which can largely be a copy / paste for most adapters. Just remember to change the connectorClass to your own, which you'll implement in the next step (below).

Implement an OMRSRepositoryConnector

Next, write an OMRSRepositoryConnector specific to your connector, which extends OMRSRepositoryConnector. This defines the logic to connect to and disconnect from your metadata repository. As such the main logic of this class will be implemented by:

...

You may want to wrap the metadata repository client’s methods with your own methods in this class as well. Generally think of this class as “speaking the language” of your proprietary metadata repository, while the next class “speaks” Egeria.

Implement an OMRSMetadataCollection

Finally, write an OMRSMetadataCollection specific to your repository, which extends OMRSMetadataCollectionBase. This can grow to be quite a large class, with many methods, but is essential for the participation of your metadata repository in a broader cohort. In particular, it is heavily leveraged by Egeria’s Enterprise Connector to federate actions against your metadata repository. As such, this is how your connector “speaks” Egeria (open metadata).

...

• a connection to be instantiated to your repository, and
• translation between your repository’s representation of metadata and the open metadata standard types.

Package your connector

...

Of course, if you intend to embed or otherwise implement your own server, the packaging mechanism will likely be different. However, as mentioned in the previous step this should provide a quick and easy initial way of testing the functionality of the connector against the core of Egeria.

Startup the OMAG Server Platform with your connector

...

Start with just the following:

Enable the OMAG Server as a repository proxy

Enable the OMAG Server as a repository proxy by specifying your canonical OMRSRepositoryConnectorProvider class name for the connectorProvider={javaClassName} parameter and POSTing to:

...

{
  "class": "Connection",
  "connectorType": {
    "class": "ConnectorType",
    "connectorProviderClassName": "org.odpi.egeria.connectors.apache.atlas.repositoryconnector.ApacheAtlasOMRSRepositoryConnectorProvider"
  },
  "endpoint": {
    "class": "Endpoint",
    "address": "{{atlas_host}}:{{atlas_port}}",
    "protocol": "http"
  },
  "userId": "{{atlas_user}}",
  "clearPassword": "{{atlas_password}}"
}

Start the server instance

Start the OMAG Server instance by POSTing to:

...

During server startup you should then see various messages related to the metadata type registration process as the open metadata types are checked against your repository. (These in turn call the methods you’ve implemented in your OMRSMetadataCollection.) You might naturally need to iron out a few bugs in those methods before proceeding further…

Test your connector's basic operations

A logical order of implementation might be:

Read operations

getEntitySummary()

... which you can test through GET to: 

http://egeriahost:8080/servers/test/open-metadata/repository-services/users/myself/instances/entity/{{guidOfEntity}}/summary

getEntityDetail()

... which you can test through GET to: 

http://egeriahost:8080/servers/test/open-metadata/repository-services/users/myself/instances/entity/{{guidOfEntity}}

getRelationshipsForEntity()

... which you can test through POST to: 

...

These are likely to require the most significant logic for any mappings / translations you’re doing between the open metadata types and your own repository. For example, with Apache Atlas these are where we translate between the Apache Atlas native types like AtlasGlossaryTerm and its representation in the Apache Atlas java client and the open metadata type GlossaryTerm and its representation through the standard OMRS interfaces.

Search operations

The other main area to then implement is searching, for example:

findEntitiesByProperty()

... which you can test through POST to: 

...

{
  "class": "EntityPropertyFindRequest",
  "typeGUID": "0db3e6ec-f5ef-4d75-ae38-b7ee6fd6ec0a",
  "pageSize": 10,
  "matchCriteria": "ALL",
  "matchProperties": {
    "class": "InstanceProperties",
    "instanceProperties": {
      "displayName": {
        "class": "PrimitivePropertyValue",
        "instancePropertyCategory": "PRIMITIVE",
        "primitiveDefCategory": "OM_PRIMITIVE_TYPE_STRING",
        "primitiveValue": "\\QEmpl\\E.*"
      }
    }
  },
  "limitResultsByClassification": [ "SpineObject" ]
}

findEntitiesByClassification()

... which you can test through POST to: 

...

{
  "class": "EntityPropertyFindRequest",
  "typeGUID": "0db3e6ec-f5ef-4d75-ae38-b7ee6fd6ec0a",
  "pageSize": 100,
  "matchClassificationCriteria": "ALL",
  "matchClassificationProperties": {
    "class": "InstanceProperties",
    "instanceProperties": {
      "scope": {
        "class": "PrimitivePropertyValue",
        "instancePropertyCategory": "PRIMITIVE",
        "primitiveDefCategory": "OM_PRIMITIVE_TYPE_STRING",
        "primitiveValue": "*local*"
      }
    }
  }
}

findEntitiesByPropertyValue()

... which you can test through POST to: 

...

Once you have those working, it should be relatively easy to go back and fill in areas like the other TypeDef-related methods, to ensure your connector can participate appropriately in a broader open metadata cohort.

Write operations

While the ordering above is necessary for all connectors, if you’ve decided to also implement write operations for your repository there are further methods to override. These include:

...

If you are only implementing a read-only connector, these methods can be left as-is and the base class will indicate they are not supported by your connector.

Add an event mapper

The event mapper connector enables events from an existing metadata repository to distribute changes to metadata to the rest of the metadata repositories who are members of the same OMRS cohort. It is not a mandatory component: as long as your connector can “speak” Egeria through an OMRSMetadataCollection it can participate in an open metadata cohort via the Enterprise Connector. However, if your metadata repository already has some kind of event or notification mechanism, the event mapper can be an efficient addition to participating in the broader open metadata cohort.

Within the same adapter Maven module, perhaps under a new sub-package like ...eventmapper, implement the following:

Implement an OMRSRepositoryEventMapperProvider

Start by writing an OMRSRepositoryEventMapperProvider specific to your connector, which extends OMRSRepositoryConnectorProviderBase. The connector provider is a factory for its corresponding connector. Much of the logic needed is coded in the base class, and therefore your implementation really only involves defining the connector class and setting this in the constructor.

...

Note that you’ll need to define a unique GUID for the connector type, and a meaningful name and description. Really all you then need to implement is the constructor, which can largely be a copy / paste for most adapters. Just remember to change the connectorClass to your own, which you’ll implement in the next step (below).

Implement an OMRSRepositoryEventMapper

Next, write an OMRSRepositoryEventMapper specific to your connector, which extends OMRSRepositoryEventMapperBase and implements VirtualConnectorExtension and OpenMetadataTopicListener. This defines the logic to pickup and process events or notifications from your repository and produce corresponding OMRS events. As such the main logic of this class will be implemented by:

...

To add the event mapper configuration to the OMAG Server Platform configuration you started with above, add:

Configure the cohort event bus

This should be done first, before any of the other configuration steps above, by POSTing to:

...

{
  "producer": {
    "bootstrap.servers":"kafkahost:9092"
  },
  "consumer": {
    "bootstrap.servers":"kafkahost:9092"
  }
}

Configure the event mapper

This can be done nearly last, after all of the other configuration steps above but still before the start of the server instance. Specify your canonical OMRSRepositoryEventMapperProvider class name for the connectorProvider={javaClassName} parameter and connection details to your repository’s event source in the eventSource parameter by POSTing to:

...

http://egeriahost:8080/open-metadata/admin-services/users/myself/servers/test/local-repository/event-mapper-details?connectorProvider=org.odpi.egeria.connectors.apache.atlas.eventmapper.ApacheAtlasOMRSRepositoryEventMapperProvider&eventSource=atlashost:9027

Test your connector's conformance

...