Central Component for #datadrivenarchitecture: Real-time Insights powered by Reactive Programming #netflix

Can help with:

  • testing
  • debugging
  • security

Solution:

  • log everything

Problems:

  • so much data
  • so many devices
  • not feasible to save to elasticsearch first (real time!)

Stream analysis with reactive programming

In a data driven architecture the processors for the high performance message bus benefit from being written in Rx.

Who use es it?

users of rxjs: netflix, google, facebook and more

Existing Solutions:

Differences

  • Mantis is designed for operational use cases where message guarantee levels vary by jobs. So, some jobs can choose at-most once guarantees while others choose at least once guarantees via Kafka. We are able to saturate the NIC on the servers for operational use case with very little CPU usage.
  • Bulit-in back pressure that allows Mantis to seamlessly switch between push, pull or mixed modes based on the type of data sources
  • Support a mix of long running perpetual analysis jobs along with user triggered short lived queries in a common cluster
  • Since the volume of data to be processed at Netflix varies tremendously by time of day being able to autoscale workers in a job based on resource consumption & the ability to scale the cluster as a whole was a key requirement. None of the existing streaming frameworks provided such support.
  • We wanted more control over how we schedule the resources so we can do smarter allocations like bin packing etc. (that also allows us to scale the jobs)
  • deep integration with Netflix ecosystem that allows filtering event stream at the source of data.
    among others

Sources:

DOING THE REACT TUTORIAL | LIVE CODING #1 | BUILDTHEFUTURE

Modern application are delivered with web technologies – no discussion. Congratulations Javascript you have won. And with React and it’s virtual DOM the performance increased to a level that is close to native. So I decided to get my hands dirty and do the official react tutorial. Hope you enjoy this guys.

Reduce Poverty in Inner Cities

An Overview of which factors are making cities smarter and why, what’s possible and how and what we can expect, and when.

Smarter cities are providing a myriad of benefits for their people, businesses and governments:
*an improved quality of life,
*sustained economic vitality,
*and intense communication through social media.


Michael Dixon leads IBM’s strategy and its implementation for Smarter Cities around the world. Prior to a brief period as the Managing Partner for IBM’s consulting and systems integration business (GBS) in Central/Eastern Europe in 2011, Michael was the Vice President of Public Sector for GBS in the Growth Markets, leading teams focused on supporting client initiatives in Government, Healthcare Payers and Providers and Education across Asia Pacific, the Middle East, Africa, Central / Eastern Europe and Latin America. Client success in numerous projects resulted with IBM providing strategic advice, business change and process transformation to improve services and deliver cost efficiencies.
Before joining IBM Michael completed a BBSc at La Trobe University, an MSc at the University of Melbourne and a PhD at the University of Alberta. Michael is an Australian and spends his time in Melbourne, New York and all parts of the world.

BETA [WIP] Execution Framework for Daily Workload

As I described I think we all need to be agile and pivoting like startups to make the best out of our resource. When we do we have the chance to change the world. One small task at a time.

The following presentation gives you an overview how J.D. Meier, a project manager at microsoft adapted Agile to a productivity method.

http://www.30daysofgettingresults.com/2011/10/day-7-setup-boundaries-and-buffers.html

I made the workflow somewhat more concrete:

For me the core value is that a task should create something tangible. Only code in the repository exist. (No speed limits!)

  1. COLLECT everything into an Inbox: This is very important as it allows you to have an Inbox Zero, stops your from the task of keeping things in your mind but just dump it into your Inbox (For me this is an Evernote Folder INBOX).
  2. ORGANIZE:
    1. If it is not related to a task delete it. Or if possible don’t even put it into the inbox. The management of an repository of information is to expensive. Use social networks.
    2. If it can be done in less than 2 min do immediately and archived item in BACKLOG (add a reminder and instantly mark as done).
    3. Else create a project for it and answer two questions What is my goal? Why is this important to me? If it is not important it I add an reminder and directly goes into backlog. If it is important I tag it with importantx as well as the area where it belongs.
    4. Monday Vision: Check your Hierarchy of GOAL & Backlog what is relevant for this week, and select 3 result for this week: “If this were Friday, what are three results I want under my belt?”
    5. Daily Win: Check the Monday Vision for 3 result of today: ““What are the three things I really want to accomplish today?”
  3. EXECUTE: Set of values, experiences and best practices.
  4. REVIEW: Daily scrum, friday reflection

The most important part are the VALUES of the doing & the executing to apply:

  • Only tangible results count (“Writing code”)
  • It is about feedback loops and iteration: If you give the product to the customer he will give you feedback. If you don’t progress you either not produce or you produce nothing good.
  • Fail fast & Done is better then perfect