Challenge
Nexa was born from a vision to leverage Watts hardware (IoT sensors, equipment, and gateways) and cloud native technologies to give customers a near-real-time view of their water system and connected devices. Nexa, in a way, is an observability platform for commercial buildings’ water systems that also provides a digital twin experience. It offers customers continuous alerting and monitoring that previously didn’t exist alongside key features such as leak detection and water shut-off capabilities.
During the project kickoff for Nexa, we spent a lot of time researching and evaluating technologies that we believed we’d need in order to build a real-time, event-driven system that scales as telemetry scales. We also knew open source was going to be a key differentiator for us to be successful launching Nexa within the time constraints. We started out with the simple things: configuration, storage, message bus, and different services satisfying core parts of the system; what was missing was the glue holding it all together.
We designed architecture ‘blueprints’ for the system, and then began plugging in technologies on the CNCF landscape that would solve our use-cases. One of the biggest questions we knowingly tried to solve for up front was: “How will this distributed architecture grow overtime and what can we do to streamline the development of an event-driven system?” – and that’s when we discovered Dapr.
Solution
Dapr immediately blew us away with its simple set of APIs that provided an abstraction over the complexities of building a distributed application. Dapr immediately offered solutions to some of our core open questions: secure service to service invocation, developer-friendly debugging, CRON jobs, pub/sub, resiliency, and so much more. It felt like we found an answer to the standard set of questions that every new distributed system faces which often results in teams rolling their own creative solutions to solve what Dapr can provide out of the box. We were able to stand up a completely distributed architecture in Azure in less than one monththanks to Dapr.
Pub/sub – The pub/sub API is a core part of our platform and a shining example of how development is truly accelerated by Dapr. The pub/sub API enabled the system to begin ingesting telemetry and publishing to a single topic which all services in our architecture subscribe and react accordingly to – with less than 50 lines of application code total! In an IoT solution like Nexa, telemetry is the heartbeat of the system; it’s used for analytics, customized rules and alerts, a real-time dashboard, a digital twin representation in the system map, and more. The best part of the solution is the underlying message provider didn’t matter. We were able to leverage both Azure IoT Hub + Event Hubs and Azure Service Bus. We are able to expand the infrastructure our system integrates with and not worry about rearchitecting the system or refactoring application code or business logic due to changes in SDKs – which is extremely powerful. Without Dapr, we’d have to pull in several different SDKs for the different messaging providers and we’d have to pull in a solution like MassTransit to handle simplifying messaging architecture, which is a .NET focused solution.
Today, we’re processing over 115 million telemetry events per month with Dapr!
Service invocation – Invoking APIs from other services has become a breeze with Dapr. Without Dapr, we’d need to manage URLs via configuration files and have special accommodation for local development vs when deployed to the cloud, which leads to a tightly coupled system. With Dapr, we’re able to securely communicate across services in a decoupled manner with mTLS out of the box. Taking advantage of Daprs retry policies, developers don’t have to worry about pulling in resiliency libraries (Polly.NET, for example) to build custom circuit breaker implementations, which is a great time saver and language-agnostic feature.
CRON – We take advantage of CRON bindings for several features, but an easy feature to highlight is our offline detection for connected devices. Without Dapr, we’d need to write and deploy custom CRON jobs, which could lead to deploying new infrastructure (Azure Functions or a new Service) or adding new open source scheduling dependencies, similar to Hangfire or Quartz.NET, and becoming familiar with the accompanying SDK. With Dapr, we implement a simple API that is invoked at the desired schedule as defined in the cron binding configuration and implement the core logic of the API.
SDKs – We didn’t have to hassle with learning and implementing provider-specific SDKs – we just needed to know Dapr’s APIs!
Other bindings & APIs:
State management API – When we need to share state across several services, we leverage Dapr’s state store capabilities, which allows us to remain vendor/technology-agnostic. Instead of needing to install an SDK to talk to Redis, for example, Dapr delivers a consistent set of state store APIs that make it simple to get up and running!
Output bindings – When we need to generate an alert based off of a workflow event, we leverage Dapr’s output binding with Twilio and SendGrid.
Impact
At Watts, we’re always looking for ways to move faster and reduce cognitive load on our engineering team. Thanks to Dapr, development teams at Watts don’t face the context switching associated with abundant SDKs and the maintenance / nuances between them – we’re able to largely look at integrations through the lens of Dapr. Debugging is a breeze with a local Dapr install which gives developers the confidence they need when deploying new services as well as the hand-off between developers and the DevOps team.
Dapr has become a core technology embedded in each system we’ve built since discovering it in 2022, which includes Nexa and a re-write of our European platform called Vision+. Put simply, Dapr allows our development to focus on business logic and features that deliver a powerful digital experience and it allows our platform team to focus on other areas that Dapr streamlines. Dapr is the de facto choice when building distributed applications at Watts and has been a core part of our ability to move at a rapid pace, empowering us to innovate fearlessly and deliver solutions with unmatched speed and reliability.
It’s not often a team with less than 35 engineers sets out to build two brand new SaaS solutions within the same timeframe, but that’s what we did and ultimately how we uncovered the power of Dapr.
When it comes to building event-driven distributed systems, especially cloud native solutions, there are hundreds of patterns and technologies to choose from. Ironically, distributed systems are still born with a consistent set of problems to solve. To name a few:
- Secure service to service invocation
- Pub/sub
- Support for polyglot services
- App configuration / secrets support
- SDK/Provider fatigue
In a typical .NET application, you may see some of the below packages for pub/sub (MassTransit), service to service invocation with retry (Polly, App Configuration for managing service URLs, etc.), CRON jobs (which may require a new service or infrastructure), and more. Developers have options, but these options can be limiting in key areas. For example, if I have .NET and Python services running in the same environment and I wanted to use Azure Service Bus as a message broker, without Dapr I am reliant on Azure Service Bus having an SDK implemented for both languages, which is not guaranteed. We faced this scenario when experimenting with implementing our digital twin functionality in Python instead of .NET.
These options are great at what they do, but when combined together they take development teams down silos. Siloed systems can introduce context switching and additional system requirements, such as the need to implement custom retry logic across services or hand-roll security for service-to-service communication. Developers can also be burdened with deploying new infrastructure for relatively straightforward functionality like a CRON job, which further compounds the challenges imposed by a fragmented architecture. With Dapr, you get all of these out of the box!
To emphasize this point a bit further, I’d like to highlight that we migrated our hosting/compute platform and components of our architecture in Azure with zero downtime. Since deploying to production last year, we’ve fully migrated from Azure Container Apps to Azure Kubernetes Service, swapped underlying pub/sub providers for some of our high-throughput events, and more than quadrupled the amount of telemetry our system is processing without issue. This wasn’t without great planning and execution by the engineering team at Watts, but this is also the core value prop for Dapr – the application development teams were able to stay laser focused on features that provide value to users rather than swapping out SDKs or libraries because we switched message brokers, state stores, or changed domain names for our platform. You heard that right, our application code did not need to change!
For example, this was the primary change needed to switch from Azure Service Bus (left) to Azure Event Hubs (right). It literally couldn’t be easier:
This flexibility empowers development teams to select best-fit technologies without being constrained by platform-specific requirements. With Dapr, developers focus on crafting high-quality, scalable code that provides value to their users, while remaining confident that the underlying complexities are managed by a reliable, battle-tested runtime.
Dapr’s ability to abstract away the complexities of building distributed systems allows developers to focus on core business logic and delivering value to users while avoiding getting mired in the intricate details of infrastructure concerns. By providing a consistent set of APIs and building blocks for common distributed system patterns, Dapr empowers teams to move at a much faster pace. Developers can quickly spin up new microservices, integrate with a variety of data stores and pub/sub systems, and implement resiliency and observability features, all without having to change their application code and retry logic.
In summary, Dapr’s capabilities are an instant value-add that significantly reduces time-to-market for both application developers and platform teams. No other open source solution can match Dapr’s ability to accelerate the development of distributed, event-driven systems. Dapr has been a critical enabler of lightning-fast development for the App Dev teams at Watts, and will remain a cornerstone in our architecture going forward.
Read the full case study at the CNCF Website.
