Hi,

Led a team of 3 developers on a nightly job to migrate School District data to remove burden of manual data entry.Features

Problem

Our company had built a robust learning management system (LMS) but we didn’t have a way to sync to a school district’s existing system; their schools had to do it manually through Excel uploads or by filling out forms on the front end, which required maintaining data in two places. For small schools this process was time-consuming and error prone and for school districts it was a deal-breaker.

Our CEO asked me to build a system that would allow our platform to sync with any external LMS system. A discovery phase revealed most school districts model their data after IMS standards and make their data accessible through CSV files or a REST API.

Impact

Schools were unable to use the LMS for up to 3 days or longer depending on the school size. The school day would be interrupted due to manual entry errors (e.g. Student is registered in the wrong class) and that would impact the student’s, teacher’s, counselor’s and IT staff’s day trying to resolve the error.

Solution

An ETL application (Extract Transform Load) that would sftp into a school district’s server and monitor changes in CSVs and update our LMS system nightly and have the foundation for eventual REST integration.

After researching SaaS products that could handle this process for us, I realized none of them was an “end to end” solution that could handle our complicated data massaging hence I needed a custom ETL tool.

I came up with an architecture that would include a cron job to check for new CSV records nightly, migrate them to S3 for archive purposes, and trigger a lambda to handle our complex data massaging and update the necessary databases across our APIs. I additionally designed our application to allow IT users to enter their SFTP and REST credentials to instantaneous sync their data with no manual involvement on our apart.

After completing a prototype, I assigned the remaining tasks to my development team for completion.

Sprinkle of Sales

I made sure to build transaction monitoring into the scope because I learned from previous ETL jobs the importance of knowing the source of data movement. This feature could later be built into a full-blown dashboard for school districts to monitor their data pipelines. The Sales team loved this idea because we could potentially charge more for it.

Architectural Diagram

Results

Video Overview

Architected .NET middleware application to automate processes for a health foundation to coalesce three different APIs – Survey Monkey, Dynamics CRM, SharePoint.Features

Problem

We implemented a CRM integration for a health foundation to provide data visibility across their departments and automate routine workflows. However, the Advocacy department was reluctant to use the CRM and the integration needed “buy in” from them to be successful.

This integration would help other departments but how does it help us? –Advocacy Department

A discovery phase revealed that they did have many manual processes that could be automated with the CRM being a key player.

Impact

Advocacy department: Unnecessary time spent distributing, collecting and sharing the results of surveys.

Health foundation: Inability to make business decisions due to limited insight into the Advocacy department’s throughput and user base.

Solution

To replace their existing manual data collection methods, we built a custom middleware application that would leverage webhooks in their survey software (Survey Monkey) to monitor for submissions and then populate the results in the CRM. Subsequently it would generate PDF documents and store them in SharePoint with the proper association in the CRM.

I suggested we don’t just build this middleware for this standalone service, but requested for a larger engagement to allow this service to be abstracted to allow any kind of sync with an external system. The Advocacy department’s automation would be the first of many. Our COO was delighted by this suggestion because it meant a larger proposal.

Results

• Hours of manual entry saved due to middleware automating these operations.
• Costs saved on reporting tools since they were replaced with CRM reporting.
• Foundation admin were better positioned to allocate funding and see relationships within the user base once they had the Advocacy departments data.

Complexities

Some departments were hesitant to join the CRM due to data privacy; we had to carefully impliment policies to ensure all contacts were shared but not all data was visible for each contact.

Ensuring we understood the Advocacy department’s goals and workflows so the CRM brought them benefit otherwise their reluctance would thwart the CTO’s efforts to democratize the foundation’s data. We had to really flex our consulting and onboarding skills to unify all departments to sharing data and using one system.

Led a team of 3 developers on a nightly job to migrate School District data to remove burden of manual data entry.Features

Problem

Our company had built a robust learning management system (LMS) but we didn’t have a way to sync to a school district’s existing system; their schools had to do it manually through Excel uploads or by filling out forms on the front end, which required maintaining data in two places. For small schools this process was time-consuming and error prone and for school districts it was a deal-breaker.

Our CEO asked me to build a system that would allow our platform to sync with any external LMS system. A discovery phase revealed that most school districts model their data after IMS standards and make their data accessible through CSV files or a REST API.

Impact

Schools were unable to use the LMS for up to 3 days or longer depending on the school size. The school day would be interrupted due to manual entry errors (e.g. Student is registered in the wrong class) and that would impact the student’s, teacher’s, counselor’s and IT staff’s day trying to resolve the error.

Solution

An ETL application (Extract Transform Load) that would sftp into a school district’s server and monitor changes in CSVs and update our LMS system nightly and have the foundation for eventual REST integration.

After researching SaaS products that could handle this process for us, I realized none of them was an “end to end” solution that could handle our complicated data massaging hence I needed a custom ETL tool.

I came up with an architecture that would include a cron job to check for new CSV records nightly, migrate them to S3 for archive purposes, and trigger a lambda to handle our complex data massaging and update the necessary databases across our APIs. I additionally designed our application to allow IT users to enter their SFTP and REST credentials to instantaneous sync their data with no manual involvement on our apart.

After completing a prototype, I assigned the remaining tasks to my development team for completion.

Sprinkle of Sales

I made sure to build transaction monitoring into the scope because I learned from previous ETL jobs the importance of knowing the source of data movement. This feature could later be built into a full-blown dashboard for school districts to monitor their data pipelines. The Sales team loved this idea because we could potentially charge more for it.

Architectural Diagram

Results

Video Overview

Architected a microservice to offer multiple apps document storage where the User decides how their data can be used.Features

Problem

Our company was building a document sharing application for healthcare and education providers to gather research from marginalized communities. I repeatedly heard from stakeholders that users in the community were apprehensive about sharing data. They didn’t want it shared with a 3rd party without their consent, worse-still, shared with government authorities due to poor experiences with law enforcement officials.

Impact

The providers were not getting “buy in” from their users hence application use was minimal or certain features that required users to share personal information was being ignored.

Solution

Results

Video Overview

Architected .NET middleware application to automate processes for a health foundation to coalesce three different APIs – Survey Monkey, Dynamics CRM, SharePoint.Features

Problem

Our company was midway through a CRM integration for one of the largest health foundations when they hit an impasse. One department was reluctant to adopt the CRM because they felt it wasn’t useful. The project’s success was contingent on every department using the CRM seeing as the ultimate goal was to provide data visibility across the organization.

If they don’t join, it’ll thwart the whole project’s effort to democratize our data. – Foundation CTO

The Advocacy department’s primary responsibility was surveying the foundation’s user base for outreach feedback. During discovery I learned their workflow had many manual steps that were ripe for automation and they were using 3rd party reporting and document sharing tools to compensate for a unified system.

Solution

To replace their existing manual data collection methods, I built a custom middleware application that would leverage webhooks in their survey software (Survey Monkey) to monitor for submissions and run this workflow:

1. Create/update contacts in the CRM based on survey responses
2. Create relevant meta entities and relationships in CRM
3. Generate a PDF document of their results and associate with contact
4. Create SharePoint folders to store generated PDFs
5. Notify the Advocacy department of contact changes

To handle high survey throughput we utilized concurrent processing to process responses 5x faster. I additionally made a custom dashboard for them to monitor survey activity and contact changes to provide full visibility and reporting on their user base.

Sprinkle of Salesmanship

I suggested we don’t just build this middleware as a standalone service; I proposed a larger engagement to allow this service to be abstracted to allow any kind of sync with an external system. The Advocacy department’s automation would be the first of many. Our COO was delighted by this suggestion because it meant a larger proposal.

Results

• Hours of manual entry saved for the Advocacy department
• Costs saved on 3rd reporting tools since they were eliminated
• Enhanced visibility into user base via Dashboard
• C-level personnel better positioned to make business decisions due to all departments using one system.

Complexities

Some departments were hesitant to join the CRM due to data privacy; I had to carefully implement policies to ensure all contacts were shared but not all data was visible for each contact.

Ensuring we understood the Advocacy department’s goals and workflows so the CRM brought them benefit otherwise their reluctance would thwart the CTO’s efforts to democratize the foundation’s data. I had to really flex my consulting and onboarding skills to unify all departments to sharing data and using one system.