FICO’s platform engineering team is constantly researching new tools to help accelerate delivery of solutions to their customers. In this presentation, Hugo Smitter, Principal Platform Architect at FICO, reviews the design of the tech stack that enables their teams to write sophisticated automation pipelines. Learn how they leverage Crossplane and Dapr to build Composition pipelines for increased deployment flexibility and velocity.

For detailed code examples and the recorded demo mentioned in this talk, visit the GitHub repository.

‍

In this session:

- Overview of the FICO® Platform and strategic business objectives driving our tool selection.

- Crossplane Functions for dynamic resource generation, database interactions, secret management and more.

- Combine Dapr’s building blocks with Crossplane to streamline your DevOps workflows to increase deployment flexibility and velocity.

- Building Anything-as-Code (AaC) pipelines using Crossplane’s Composition Functions aided by Dapr to separate user logic from boilerplate code.

- Gain technical insights and practical knowledge on new tools to increase velocity and flexibility of your team’s workflows and AaC deployments.

‍

‍

Leveraging Crossplane Functions and Dapr for Scalable Platform Engineering at FICO

Platform engineering is rapidly evolving to address the complexities of modern application development. Organizations are moving from isolated point solutions to unified, scalable platforms to improve operational efficiency and offer nuanced solutions to customers. This article presents insights from FICO’s journey to build a unified platform, focusing on the integration of Crossplane Functions and Dapr to overcome architectural and operational challenges.

FICO’s Evolution Toward a Platform-Centric Approach

FICO, widely known for its credit scoring system, has evolved significantly over its 68-year history. While its score-related services remain central, FICO now offers a comprehensive ecosystem of solutions enabling clients to make faster, smarter decisions. However, this diversification has introduced challenges in unifying disparate systems under a cohesive platform architecture.

A key realization for FICO was that identifying and leveraging shared components across independent business units could yield a more robust, monetizable platform. Central to this evolution is the adoption of event-driven architectures and the migration from Apache Kafka to Apache Pulsar as the event middleware.

To avoid lock-in with a specific technology, FICO implemented an abstraction layer leveraging Dapr. This decision was pivotal in maintaining flexibility and ensuring that future migrations or integrations could proceed seamlessly.

Integrating Crossplane Functions and Dapr

FICO’s platform engineering team identified an opportunity to combine the capabilities of Crossplane and Dapr. Crossplane, a Kubernetes-native platform orchestrator, enables the declarative management of cloud resources. Its introduction of Functions, which allow programmatic resource generation and pipelines, opens new possibilities for platform engineers.

Dapr, the Distributed Application Runtime, complements Crossplane by offering modular building blocks for common distributed system concerns, such as state management, pub/sub messaging, and secret handling. Together, Crossplane Functions and Dapr provide a powerful toolkit for building, managing, and scaling complex systems.

Challenges in Adopting Crossplane Functions

1. Reconciliation Loops

Crossplane functions operate within Kubernetes’ reconciliation loop. While this ensures desired state consistency, it introduces complexity. Functions may be invoked multiple times during resource creation, requiring careful design to avoid unintended side effects.

2. Separation of Concerns

Functions provide immense flexibility but can lead to “spaghetti code” if not managed properly. Maintaining clean separation between infrastructure concerns and business logic is critical.

3. Dynamic Resource Generation

Crossplane functions enable dynamic resource creation, pulling templates or configurations from external sources. However, this disrupts traditional policy validation workflows, as upstream CI/CD pipelines may not have full visibility into dynamically generated resources.

Dapr’s Role in Enhancing Crossplane Functions

Dapr simplifies the implementation of distributed systems by abstracting cross-cutting concerns. FICO’s use of Dapr focused on the following key building blocks:

• State Store: Used to persist state across resource generation workflows.

• Distributed Lock: Prevents redundant processing during reconciliation loops.

• Service Invocation: Facilitates communication between Crossplane functions and external services.

• Pub/Sub Messaging: Supports event-driven notifications for resource completion.

By leveraging Dapr’s modularity, FICO reduced complexity in function development while gaining the ability to swap underlying technologies without rewriting code.

Proof of Concept: Policy Validation with Crossplane and Dapr

One of FICO’s significant challenges was validating policy compliance during resource generation. Traditional policy engines, such as Kyverno, operate either at the CI/CD stage or admission control. However, dynamically generated resources often bypass these validation points.

To address this, FICO introduced a Policy Validator Function within the Crossplane pipeline:

1. Resource Validation: The function gathers dynamically generated resources at the end of the pipeline.

2. Policy Fetching: Using Dapr’s service invocation, the function retrieves applicable policies from a secure repository.

3. Policy Evaluation: Policies are applied to the gathered resources using Kyverno’s CLI.

4. Feedback Loop: Results are reported back to the Crossplane composition, marking success or failure.

This integration ensures that all resources—static or dynamically generated—are compliant with organizational policies before final deployment.

Simplified Development with Dapr

Dapr supports both Kubernetes and local development environments. On Kubernetes, Dapr operates as a sidecar or daemon set, injecting cross-cutting capabilities into applications. For local development, Dapr runs as a standalone process, enabling developers to test Crossplane functions with full Dapr capabilities on their laptops.

Key steps in integrating Dapr with Crossplane include:

• Adding HTTP REST support in Crossplane functions to interact with Dapr.

• Configuring Dapr through Kubernetes manifests for production deployments.

• Merging Dapr and Crossplane runtime configurations for seamless local development.

Future Directions

FICO’s integration of Crossplane and Dapr lays the foundation for further enhancements:

1. Transactional Outbox Pattern: Using Dapr to achieve atomic consistency between state updates and pub/sub messaging.

2. Advanced Workflow Automation: Leveraging Dapr’s workflow building block to orchestrate multi-step processes.

3. Scalable Multi-Tenant Architectures: Extending support for managing tenant-specific resources dynamically.

The synergy between Crossplane and Dapr represents a significant step forward for platform engineering. By integrating these tools, FICO has built a more flexible, scalable, and policy-compliant platform that aligns with the needs of modern distributed systems.

This approach demonstrates how combining Kubernetes-native orchestration with a modular runtime like Dapr can drive innovation in platform engineering while ensuring governance and operational excellence.

‍