My Data Driven Applications Stack

The migration from functional applications and architectures to data centric / flow / reactive architectures reminds me of the industrial revolution. Back then after the invention of electricity we just replaced stream with electricity and the result where underwhelming. This change after we redesign the manufacturing space according to the processes (which was no possible as energy could now be transported to the machines). In this article I try to clarify for myself how a architecture looks like when we focus fully on data (which is the key resource of today’s companies).

First an overview of all components with links to more details and in over time links to the sample application where I will build this in.

Work in Progress


Data Flow Oriented (Reactive) Interfaces / Sensors / Actors:

Types: Web / Mobile / IoT (Car / Wearables / Robots / Devices)
– Internal: React, Redux, RxJS
– Integration: Kafka Client, REST (best practices), JSON

Services / Cognitive Functions / Backends:

– Internal: RxJS/Java, NodeJS, Scala/Java, Akka, Play, Python, Spark, Tensorflow
– Integration: Kafka (Messages), DFS / Apache Nifi (Payload), Formats(JSON, Avro)

Build Processes

– Everything in containers (including DB, Analytics …)
Containers: Docker/Kubernetes/Helm
Continuous Integration: Jenkins, JUnit, A/B Testing, Code Coverage


  • internal/external open sourcing (owner/ pull request / reviews ! / forks  -> no central components)
  • requirement analysis / inter team / business / it collaboration (consumer driven contracts)
  • business UIs (pega needs that too … business people … new need a pega course …)


  • security by design ( part of the automated dev pipeline (check for licenses, check for container vulnerabilities) -> warning noch deal breakers) (JWT …)

Architecture Guidelines:

  • Teams are free inside their application, but limited in the communication between applications
  • Services should be structured according to their bounded context (domain driven design)
  • Migrate the old world by isolating/proxy and abstracting the interfaces

Leave a Reply