My Data Driven Applications Stack

The migration from functional applications and architectures to data centric / flow / reactive architectures reminds me of the industrial revolution. Back then after the invention of electricity we just replaced stream with electricity and the result where underwhelming. This change after we redesign the manufacturing space according to the processes (which was no possible as energy could now be transported to the machines). In this article I try to clarify for myself how a architecture looks like when we focus fully on data (which is the key resource of today’s companies).

First an overview of all components with links to more details and in over time links to the sample application where I will build this in.

Work in Progress


Data Flow Oriented (Reactive) Interfaces / Sensors / Actors:

Types: Web / Mobile / IoT (Car / Wearables / Robots / Devices)
– Internal: React, Redux, RxJS
– Integration: Kafka Client, REST (best practices), JSON

Services / Cognitive Functions / Backends:

– Internal: RxJS/Java, NodeJS, Scala/Java, Akka, Play, Python, Spark, Tensorflow
– Integration: Kafka (Messages), DFS / Apache Nifi (Payload), Formats(JSON, Avro)

Build Processes

– Everything in containers (including DB, Analytics …)
Containers: Docker/Kubernetes/Helm
Continuous Integration: Jenkins, JUnit, A/B Testing, Code Coverage


  • internal/external open sourcing (owner/ pull request / reviews ! / forks  -> no central components)
  • requirement analysis / inter team / business / it collaboration (consumer driven contracts)
  • business UIs (pega needs that too … business people … new need a pega course …)


  • security by design ( part of the automated dev pipeline (check for licenses, check for container vulnerabilities) -> warning noch deal breakers) (JWT …)

Architecture Guidelines:

  • Teams are free inside their application, but limited in the communication between applications
  • Services should be structured according to their bounded context (domain driven design)
  • Migrate the old world by isolating/proxy and abstracting the interfaces

#IBMIntegrationBus modern #devops #automation setup – testing, containers, codecoverage

Currently I use the following IIB Tools to automate as much as possible and to be scalable in (multiple 😉 ) seconds, if you have tools I should have a look at please share with me:

#failed to make it to the best 40% at a hackathon. #frustrating #workharder

Today the cognitive hackathon at IBM IoT ended and my personal result was: bad. I did not make it to the top 3 from 8 teams. That was frustrating and I was disappointed with myself. So …

The task was to build a cognitive car concierge services. Our result was a chatbot that had
– an active voice interface (you could have a dialog without clicking all the time to speak),
– predictive notifications (about your fuel) based on your driving target and the real time vehicle data,
– it also would find the nearest Drive Now Vehicle,
– it could turn on your car (simulated by a hue light, but this could be a call to the CAN Bus of a real car)
– it could interact with google maps to find the best route.
– it had JWT based authentication in every micro service.
– it could schedule calls with your personal call center agent (you don’t need to call, but based on your problem and your customer relationship data and the calendar of the agent the best spot would be found) as well as rescheduling

Well in the end I did not perform (to my own expectations), but at least I learned somethings:
reality-check: I am better then 1 year ago, but still mediocre compared to the best in the field. I will have to work harder.
frontend is important: My failure was missing a WOW frontend, I should always keep practicing at least some frontend in the future
i don’t present very well: I did not do this for some time and I seemed totally of my game, like a 4 year old. I have to get this fixed fast. I have to do more public vlogs and learning videos, go to hackthons where I present.
it is all show: Technology does not count, as long as you have a great mockup that seems interactive it is enough. If your forced to use technologies you don’t like, just change the game and present a mockup how you would do it there but focus on the tech you like.
starter kits matter: I definitely have to improve my starter kit to be more extendible and to be better on the eye also the analytics/data component is essential.
only consider jobs where you have people better than you: Even though I failed miserably overall, it seems from a tech perspective I was one of the better ones. And in my job I focus on learning so I would need to be in a team where most of the people are better than me.

  • Now I will just put some EDM on my ears and work hard and next time, which will then be my second hackathon, I will be better.*

Let’s get down to implement the persistent high performance message bus integration (based on the work Blizzard has done) into the starter kit.


  • Problem: REST is a standard way to communicate with servers to retrieve data. It provides a specification based on the entities that we have present in our database. When done correctly, it can be more than adequate. When done wrong it can be a living hell.
  • Solution:
    • Implement HATEOAS [what is HATEOAS?] (Hypermedia as the Engine of Application State), and get a nice system that is flexible and easy enough to work with
    • more simple alternatives like GraphQL
      • GraphQL is an open-source project from Facebook that presents power with a simple idea… Instead of the application server defining the structure of responses, the client is given power and flexibility to request the data that it needs. GraphQL responses are tailored to the specific use case that the client is implementing, eliminating wasteful data transfer and providing future-proofing your API for use cases that your application hasn’t even encountered yet.
      • Is GraphQL a flash in the pan? Technology is hard to predict, but Github’s recent preview launch of their GraphQL API is super encouraging, and an interesting study.
  • Open Questions:
    • How to combine GraphQL with a Event Driven backend?
      • One query actually might indicate multiple events …
      • GraphQL might be a good candidate to implement Event Sourcing/CQRS pattern. Just the fact that GraphsQL allows two different query types – Queries and Mutation is a conceptual direct map to the basics of the Event Sourcing pattern of separating the reads and writes, gives a good foundation to explore this pattern, alongside other advantages. [link]

#FutureOfWork: The impact of #AI on the workplace and employee engagement

• The digital workplace merges work and life — a virtual space with applications, services and information on demand. For users, this means access to the technology they need, when they need it, on whichever device they prefer to use.

• Employees expect their enterprise systems to be as engaging, exciting and intuitive as consumer devices. Technology research company Gartner calls this a shift from technology-literate people to people-literate technology.

• Companies now have more exquisitely detailed data about how their products and services are used than they ever had before, thanks to a vast network of sensors and advanced analytic tools.

• Cognitive systems can parse all that data and learn what employees need to do their job better — even if they don’t yet know it themselves. Cognitive systems will deliver the ability to visualize vast amounts of data, curated and analyzed, for a unique task and a unique user.

• The workplace of the future will embrace emerging new cognitive and analytic capabilities. These tools can provide insights into how employees engage most effectively, what the best technologies are for each task and for each individual, and help provide a seamless work environment — an environment that will help to attract and retain the best talent.