Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SDK functionality to make it easier to deserialize/consume EventGrid events #2426

Merged
merged 7 commits into from
Sep 27, 2018
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
/**
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for
* license information.
*/
package com.microsoft.azure.eventgrid.customization;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.microsoft.azure.eventgrid.models.EventGridEvent;
import com.microsoft.azure.management.apigeneration.Beta;
import com.microsoft.azure.serializer.AzureJacksonAdapter;
import com.microsoft.rest.protocol.SerializerAdapter;

import java.io.IOException;
import java.lang.reflect.Field;
import java.lang.reflect.Type;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;

/**
* The type that can be used to de-serialize eventgrid events.
*/
@Beta
public class EventGridSubscriber {
/**
* The default adapter to be used for de-serializing the events.
*/
private final AzureJacksonAdapter defaultSerializerAdapter;
/**
* The map containing user defined mapping of eventType to Java model type.
*/
private Map<String, Type> eventTypeToEventDataMapping;

/**
* Creates EventGridSubscriber with default de-serializer.
*/
@Beta
public EventGridSubscriber() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, will add it.

this.defaultSerializerAdapter = new AzureJacksonAdapter();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the Data property contains polymorphic types, will this handle that as well, or do we need some special logic for it? In the C# case, we do something like:

       // Note: If any of the events have polymorphic data, add converters for them here.
        // This enables the polymorphic deserialization for the event data.
        // For example, MediaJobCompletedEventData's JobOutput type is polymorphic 
        // based on the @odata.type property in the data.

        // Example usage: jsonSerializer.Converters.Add(new PolymorphicDeserializeJsonConverter<JobOutput>("@odata.type"));

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the tests, I do see you are supporting this, which is great! However, it wasn't quite clear to me whether this is handled automatically, or whether like in C# we need to express something like the above where we add the PolymorphicDeserializeJsonConverter explicitly.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

De-serialization of polymorphic types are handled using Jackson annotations. See this models

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks. For 1st party / system event data that contains polymorphic fields, will we have to annotate in the same way (on top of the default autorest generated class)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For system events - if discriminator is defined in swagger for corresponding event models then auto rest will generate these annotations, no need to modify the generated models.

For custom events - User has to annotate their model as shown in the tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good, thanks!

this.eventTypeToEventDataMapping = new HashMap<>();
}

/**
* Add a custom event mapping. If a mapping with same eventType exists then the old eventDataType is replaced by
* the specified eventDataType.
*
* @param eventType the event type name.
* @param eventDataType type of the Java model that the event type name mapped to.
*/
@Beta
public void putCustomEventMapping(final String eventType, final Type eventDataType) {
if (eventType == null || eventType.isEmpty()) {
throw new IllegalArgumentException("eventType parameter is required and cannot be null or empty");
}
if (eventDataType == null) {
throw new IllegalArgumentException("eventDataType parameter is required and cannot be null");
}
this.eventTypeToEventDataMapping.put(canonicalizeEventType(eventType), eventDataType);
}

/**
* Get type of the Java model that is mapped to the given eventType.
*
* @param eventType the event type name.
* @return type of the Java model if mapping exists, null otherwise.
*/
@Beta
public Type getCustomEventMapping(final String eventType) {
if (!containsCustomEventMappingFor(eventType)) {
return null;
} else {
return this.eventTypeToEventDataMapping.get(canonicalizeEventType(eventType));
}
}

/**
* @return get all registered custom event mappings.
*/
@Beta
public Set<Map.Entry<String, Type>> getAllCustomEventMappings() {
return Collections.unmodifiableSet(this.eventTypeToEventDataMapping.entrySet());
}

/**
* Removes the mapping with the given eventType.
*
* @param eventType the event type name.
* @return true if the mapping exists and removed, false if mapping does not exists.
*/
@Beta
public boolean removeCustomEventMapping(final String eventType) {
if (!containsCustomEventMappingFor(eventType)) {
return false;
} else {
this.eventTypeToEventDataMapping.remove(canonicalizeEventType(eventType));
return true;
}
}

/**
* Checks if an event mapping with the given eventType exists.
*
* @param eventType the event type name.
* @return true if the mapping exists, false otherwise.
*/
@Beta
public boolean containsCustomEventMappingFor(final String eventType) {
if (eventType == null || eventType.isEmpty()) {
return false;
} else {
return this.eventTypeToEventDataMapping.containsKey(canonicalizeEventType(eventType));
}
}

/**
* De-serialize the events in the given requested content using default de-serializer.
*
* @param requestContent the request content in string format.
* @return De-serialized events.
*
* @throws IOException
*/
@Beta
public EventGridEvent[] deserializeEventGridEvents(final String requestContent) throws IOException {
return this.deserializeEventGridEvents(requestContent, this.defaultSerializerAdapter);
}

/**
* De-serialize the events in the given requested content using the provided de-serializer.
*
* @param requestContent the request content as string.
* @param serializerAdapter the de-serializer.
* @return de-serialized events.
* @throws IOException
*/
@Beta
public EventGridEvent[] deserializeEventGridEvents(final String requestContent, final SerializerAdapter<ObjectMapper> serializerAdapter) throws IOException {
EventGridEvent[] eventGridEvents = serializerAdapter.<EventGridEvent[]>deserialize(requestContent, EventGridEvent[].class);
for (EventGridEvent receivedEvent : eventGridEvents) {
if (receivedEvent.data() == null) {
continue;
} else {
final String eventType = receivedEvent.eventType();
final Type eventDataType;
if (SystemEventTypeMappings.containsMappingFor(eventType)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: can we do something like if (SystemEventTypeMappings.containsMappingFor(eventType) { eventDataType = SystemEventTypeMappings.getMapping(EventType) } else { eventDataType = getCustomEventMapping(eventType);} and then actually call deserialize & setEventData just once outside the if/else block.

Copy link
Contributor

@kalyanaj kalyanaj Sep 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For custom events, the Data property can be of any type (i.e. object, array, or any other primitive type), so we would need some special handling for it. For C#, the way we handle it is covered in 185-207 of https://github.com/Azure/azure-sdk-for-net/blob/psSdkJson6/src/SDKs/EventGrid/DataPlane/Microsoft.Azure.EventGrid/Customization/EventGridSubscriber.cs. Not sure if the current logic automatically handles that case as well, wanted to double-check.

Copy link
Member Author

@anuchandy anuchandy Sep 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. sounds good, will update the code as suggested.
  2. yes, no special handing required for primitives and arrays. we do have tests covering these scenarios.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The primitive and array tests are:

eventDataType = SystemEventTypeMappings.getMapping(eventType);
} else if (containsCustomEventMappingFor(eventType)) {
eventDataType = getCustomEventMapping(eventType);
} else {
eventDataType = null;
}
if (eventDataType != null) {
final String eventDataAsString = serializerAdapter.serializeRaw(receivedEvent.data());
final Object eventData = serializerAdapter.<Object>deserialize(eventDataAsString, eventDataType);
setEventData(receivedEvent, eventData);
}
}
}
return eventGridEvents;
}

private static void setEventData(EventGridEvent event, final Object data) {
// This reflection based way to set the data field needs to be removed once
// we expose a wither in EventGridEvent to set the data. (Check swagger + codegen)
try {
Field dataField = event.getClass().getDeclaredField("data");
dataField.setAccessible(true);
dataField.set(event, data);
} catch (NoSuchFieldException nsfe) {
throw new RuntimeException(nsfe);
} catch (IllegalAccessException iae) {
throw new RuntimeException(iae);
}
}

private static String canonicalizeEventType(final String eventType) {
if (eventType == null) {
return null;
} else {
return eventType.toLowerCase();
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
/**
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for
* license information.
*/
package com.microsoft.azure.eventgrid.customization;

import com.microsoft.azure.eventgrid.models.ContainerRegistryImageDeletedEventData;
import com.microsoft.azure.eventgrid.models.ContainerRegistryImagePushedEventData;
import com.microsoft.azure.eventgrid.models.EventHubCaptureFileCreatedEventData;
import com.microsoft.azure.eventgrid.models.IotHubDeviceConnectedEventData;
import com.microsoft.azure.eventgrid.models.IotHubDeviceCreatedEventData;
import com.microsoft.azure.eventgrid.models.IotHubDeviceDeletedEventData;
import com.microsoft.azure.eventgrid.models.IotHubDeviceDisconnectedEventData;
import com.microsoft.azure.eventgrid.models.MediaJobStateChangeEventData;
import com.microsoft.azure.eventgrid.models.ResourceActionCancelData;
import com.microsoft.azure.eventgrid.models.ResourceActionFailureData;
import com.microsoft.azure.eventgrid.models.ResourceActionSuccessData;
import com.microsoft.azure.eventgrid.models.ResourceDeleteCancelData;
import com.microsoft.azure.eventgrid.models.ResourceDeleteFailureData;
import com.microsoft.azure.eventgrid.models.ResourceDeleteSuccessData;
import com.microsoft.azure.eventgrid.models.ResourceWriteCancelData;
import com.microsoft.azure.eventgrid.models.ResourceWriteFailureData;
import com.microsoft.azure.eventgrid.models.ResourceWriteSuccessData;
import com.microsoft.azure.eventgrid.models.ServiceBusActiveMessagesAvailableWithNoListenersEventData;
import com.microsoft.azure.eventgrid.models.ServiceBusDeadletterMessagesAvailableWithNoListenersEventData;
import com.microsoft.azure.eventgrid.models.StorageBlobCreatedEventData;
import com.microsoft.azure.eventgrid.models.StorageBlobDeletedEventData;
import com.microsoft.azure.eventgrid.models.SubscriptionDeletedEventData;
import com.microsoft.azure.eventgrid.models.SubscriptionValidationEventData;
import com.microsoft.azure.management.apigeneration.Beta;

import java.lang.reflect.Type;
import java.util.HashMap;
import java.util.Map;

/**
* Mapping of system event type name to corresponding type of the Java model.
*/
@Beta
final class SystemEventTypeMappings {
/**
* The map containing system eventType to Java model type mapping.
*/
private static Map<String, Type> systemEventMappings;

static {
systemEventMappings = new HashMap<>(); // key: eventType, Value:eventDataType
//
// ContainerRegistry events.
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.CONTAINER_REGISTRY_IMAGE_PUSHED_EVENT), ContainerRegistryImagePushedEventData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.CONTAINER_REGISTRY_IMAGE_DELETED_EVENT), ContainerRegistryImageDeletedEventData.class);
//
// Device events.
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.IOT_HUB_DEVICE_CREATED_EVENT), IotHubDeviceCreatedEventData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.IOT_HUB_DEVICE_DELETED_EVENT), IotHubDeviceDeletedEventData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.IOT_HUB_DEVICE_CONNECTED_EVENT), IotHubDeviceConnectedEventData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.IOT_HUB_DEVICE_DISCONNECTED_EVENT), IotHubDeviceDisconnectedEventData.class);
//
// EventGrid events.
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.EVENT_GRID_SUBSCRIPTION_VALIDATION_EVENT), SubscriptionValidationEventData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.EVENT_GRID_SUBSCRIPTION_DELETED_EVENT), SubscriptionDeletedEventData.class);
//
// Event Hub Events.
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.EVENT_HUB_CAPTURE_FILE_CREATED_EVENT), EventHubCaptureFileCreatedEventData.class);
//
// Media Services events.
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.MEDIA_JOB_STATE_CHANGE_EVENT), MediaJobStateChangeEventData.class);
//
// Resource Manager (Azure Subscription/Resource Group) events.
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.RESOURCE_WRITE_SUCCESS_EVENT), ResourceWriteSuccessData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.RESOURCE_WRITE_FAILURE_EVENT), ResourceWriteFailureData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.RESOURCE_WRITE_CANCEL_EVENT), ResourceWriteCancelData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.RESOURCE_DELETE_SUCCESS_EVENT), ResourceDeleteSuccessData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.RESOURCE_DELETE_FAILURE_EVENT), ResourceDeleteFailureData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.RESOURCE_DELETE_CANCEL_EVENT), ResourceDeleteCancelData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.RESOURCE_ACTION_SUCCESS_EVENT), ResourceActionSuccessData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.RESOURCE_ACTION_FAILURE_EVENT), ResourceActionFailureData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.RESOURCE_ACTION_CANCEL_EVENT), ResourceActionCancelData.class);
//
// ServiceBus events.
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.SERVICE_BUS_ACTIVE_MESSAGES_AVAILABLE_WITH_NO_LISTENERS_EVENT), ServiceBusActiveMessagesAvailableWithNoListenersEventData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.SERVICE_BUS_DEADLETTER_MESSAGES_AVAILABLE_WITH_NO_LISTENER_EVENT), ServiceBusDeadletterMessagesAvailableWithNoListenersEventData.class);
//
// Storage events.
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.STORAGE_BLOB_CREATED_EVENT), StorageBlobCreatedEventData.class);
systemEventMappings.put(canonicalizeEventType(SystemEventTypes.STORAGE_BLOB_DELETED_EVENT), StorageBlobDeletedEventData.class);
}

/**
* Checks if a mapping exists for the given type.
*
* @param eventType the event type.
* @return true if mapping exists, false otherwise.
*/
@Beta
public static boolean containsMappingFor(final String eventType) {
if (eventType == null || eventType.isEmpty()) {
return false;
} else {
return systemEventMappings.containsKey(canonicalizeEventType(eventType));
}
}

/**
* Get Java model type for the given event type.
*
* @param eventType the event type.
* @return the Java model type if mapping exists, null otherwise.
*/
@Beta
public static Type getMapping(final String eventType) {
if (!containsMappingFor(eventType)) {
return null;
} else {
return systemEventMappings.get(canonicalizeEventType(eventType));
}
}

private static String canonicalizeEventType(final String eventType) {
if (eventType == null) {
return null;
} else {
return eventType.toLowerCase();
}
}
}
Loading