Orchestration Frameworks for Big Data

Javier Ramos
ITNEXT
Published in
8 min readOct 13, 2020

--

Introduction

Big Data is complex, I have written quite a bit about the vast ecosystem and the wide range of options available. One aspect that is often ignored but critical, is managing the execution of the different steps of a big data pipeline. Quite often the decision of the framework or the design of the execution process is deffered to a later stage causing many issues and delays on…

--

--

Certified Java Architect/AWS/GCP/Azure/K8s: Microservices/Docker/Kubernetes, AWS/Serverless/BigData, Kafka/Akka/Spark/AI, JS/React/Angular/PWA @JavierRamosRod