Hi,

One sec...

Enter

Architected .NET middleware application to automate processes for a health foundation to coalesce three different APIs – Survey Monkey, Dynamics CRM, SharePoint.Features

Problem

Our company was midway through a CRM integration for one of the largest health foundations when they hit an impasse. One department was reluctant to adopt the CRM because they felt it wasn’t useful. The project’s success was contingent on every department using the CRM seeing as the ultimate goal was to provide data visibility across the organization.

If they don’t join, it’ll thwart the whole project’s effort to democratize our data. Foundation CTO

The Advocacy department’s primary responsibility was surveying the foundation’s user base for outreach feedback. During discovery I learned their workflow had many manual steps that were ripe for automation and they were using 3rd party reporting and document sharing tools to compensate for a unified system.

Impact

Advocacy department: Unnecessary time spent distributing, collecting and sharing the results of surveys.
Health foundation: Inability to make business decisions due to limited insight into the Advocacy department’s throughput and user base.

Solution

To replace their existing manual data collection methods, I built a custom middleware application that would leverage webhooks in their survey software (Survey Monkey) to monitor for submissions and run this workflow:

  1. Create/update contacts in the CRM based on survey responses
  2. Create relevant meta entities and relationships in CRM
  3. Generate a PDF document of their results and associate with contact
  4. Create SharePoint folders to store generated PDFs
  5. Notify the Advocacy department of contact changes

To handle high survey throughput we utilized concurrent processing to process responses 5x faster. I additionally made a custom dashboard for them to monitor survey activity and contact changes to provide full visibility and reporting on their user base.

Sprinkle of Salesmanship

I suggested we don’t just build this middleware as a standalone service; I proposed a larger engagement to allow this service to be abstracted to allow any kind of sync with an external system. The Advocacy department’s automation would be the first of many. Our COO was delighted by this suggestion because it meant a larger proposal.

Results


Complexities

Some departments were hesitant to join the CRM due to data privacy; I had to carefully implement policies to ensure all contacts were shared but not all data was visible for each contact.

Ensuring we understood the Advocacy department’s goals and workflows so the CRM brought them benefit otherwise their reluctance would thwart the CTO’s efforts to democratize the foundation’s data. I had to really flex my consulting and onboarding skills to unify all departments to sharing data and using one system.

Collaborated on an asynchronous ETL application (Extract Transform Load) to routinely migrate financial transactions for a government entity handling statewide financial data

Features:

  • Orchestrated multiple SSIS packages to asynchronously migrate ~1M financial and transactional records daily
  • Indexed relevant columns and followed best practices in table normalization to optimize query performance
  • Configured Change-Data-Capture (CDC) to monitor data changes and replace existing triggers impacting performance
  • Created user functions and staging tables to accommodate time zone differences in UTC and to track changes
  • Collaborated with Database Administrators to implement High Availability and Disaster Recovery (HADR) redundancy

I couldn’t find many tutorials for creating a simple .NET listener for receiving Webhooks.  The out-of-the-box options in .Net support services for Github, Azure and others but if the API you’re searching for isn’t offered, it’s hard to find documentation that walks you through the process.

Required Skills

This tutorial assumes familiarity with Visual Studio and REST based API calls.  It was originally adapted from this this post and tweaked to incorporate Postman and Emma.  The solution below is only for receiving API calls from JSON requests.

Required Software

Project Setup

Begin by creating a blank Web API project in Visual Studio:

Add the following line to your WebAPI Config class.

public static class WebApiConfig
{
    public static void Register(HttpConfiguration config)
    {
        // Web API routes
        config.MapHttpAttributeRoutes();
        config.Routes.MapHttpRoute(
            name: "DefaultApi",
            routeTemplate: "api/{controller}/{id}",
            defaults: new { id = RouteParameter.Optional }
        );
 
        // Initialize Generic JSON WebHook receiver
        config.InitializeReceiveGenericJsonWebHooks();
    }
}

Install the following nuget package using the package manager console to obtain the necessary Webhook classes:

Install-Package Microsoft.AspNet.WebHooks.Receivers.Generic

Add the following line of code to aid in routing the requests.  The keys below can be anything you want but must be 32 characters.

<configuration>
<appSettings>
<add key="MS_WebHookReceiverSecret_GenericJson" value="i=1388a6b0d05eca2237f10e4a4641260b0a08f3a5,z=80ad19e357b01a04fe767067df7cd31b96a844e1" />
</appSettings>
<system.web>

The WebHookHandler base class is required to receive Webhook requests.   Paste in this template code:

using Microsoft.AspNet.WebHooks;
using Newtonsoft.Json.Linq;
using System.Threading.Tasks;

namespace MyWebhook
{
      public class GenericJsonWebHookHandler : WebHookHandler
     {
        public GenericJsonWebHookHandler()
        {
           this.Receiver = "genericjson";
        }

        public override Task ExecuteAsync(string generator,                     WebHookHandlerContext context)
        {
          // Get JSON from WebHook
          JObject data = context.GetDataOrDefault<JObject>();

           if (context.Id == "i")
           {
              //You can use the passed in Id to route differently depending on source.
           }
           else if (context.Id == "z")
           {
           }

           return Task.FromResult(true);
         }
   }
}

Let’s test to see if the handler works by building the solution and copying the URL from the output.

(There won’t be a landing page cause the server doesn’t return anything).

localhost (1)

Paste this URL into Postman and append the following:

Localhost:{Your Port}/api/webhooks/incoming/genericjson/z?code=80ad19e357b01a04fe767067df7cd31b96a844e1

URL format for .Net Webhook            URL endpoint from your webconfig

Enter anything you want into the Body.  Note: You need to include something in the Body or the GenericJson config will error out.

bodyinput

Set a breakpoint and  send the request and make sure it comes through.

postman-local

Connecting to Emma

To connect to Emma you need a public facing API.  Right click the solution and select “Publish” to send the solution to Azure for hosting.  Prior to clicking “Create Profile”, click the Advanced link to set the configuration to Debug to be able to debug from the server.

publish

Attach your application to the remote server for debugging:

View ->  Cloud Explorer ->  Find your application ->  “Attach to Debugger”

Replace localhost with your new URL and add HTTPS at the beginning or it won’t work.

https://{The Azure URL youre given}/api/webhooks/incoming/genericjson/z?code=80ad19e357b01a04fe767067df7cd31b96a844e1

You can test the listener again via Postman if you like by pasting in the URL above. 

Log into your Emma account and retrieve your public API key and Account number into Postman and create your first Webhook with your credentials from above.

emma-1

The example below sends a request each time a contact record is updated within Emma.  For a full list of available Webhooks visit the Emma API documentation.

postman-webhook

Authenticate via the Authorization tab and enter your Emma credentials.

authenticate

Send the request via Postman and it should return to you the ID of your Webhook.

Go back to your application in Visual Studio and attach to the debugger.

Test the Webhook by logging into your Emma platform and updating a contact. If you’re connected to the remote server, your breakpoint should be hit within a minute or so.

In order to use the JSON object you received from Emma you need Newtonsoft and a way to deserialize the Emma request.

using Newtonsoft.Json;

public override Task ExecuteAsync(string generator,, WebHookHandlerContext context)
{     

  JObject data = context.GetDataOrDefault<JObject>();

EmmaWebhook request = JsonConvert.DeserializeObject<EmmaWebhook>(data.Root.ToString());

You need a class to map the deserialization

        public class EmmaWebhook

        {

            public string event_name { get; set; }

            public Data data { get; set; }

        }

        public class Data

        {

            public string member_id { get; set; }

            public string account_id { get; set; }
            //The Emma result can also come with a TimeStamp
            //but I didn't want to deal with a DateTime property.

        }

Update a contact in Emma.  If you’ve entered everything correctly, you should see your result in the debugger 😉

debugger

Troubleshooting

If you experience issues where Emma is not sending the request, make sure your webhook is registered.  You can see registered webhooks with the following GET request in Postman:

https://api.e2ma.net/{Your Account Id}/webhooks.

If that doesn’t help connect your Emma account to Zapier and see if Zapier is able to receive the request.  If you’re still unable to receive the request, go back into the article and make sure you did not miss any details such as using the proper key length or HTTPS.

Collaborated on a .NET Core application to assist an investment company in showcasing their members.

Features:

  • Utilized Razor views and JQuery Datatables for user interaction
  • Implemented ‘Repository Pattern’ architecture
  • Utilized Entity Framework Core as ORM
  • Maintained deployment cycle leveraging Visual Studio with Azure deployment tools
  • Maintained ongoing updates and service requests