Managing Job Deployments in Production

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Managing Job Deployments in Production

Peter Groesbeck
How are folks here managing deployments in production?

We are deploying Flink jobs on EMR manually at the moment but would like to move towards some form of automation before anything goes into production. Adding additional EMR Steps to a long running cluster to deploy or update jobs seems like the most accessible method, however getting feedback output (i.e. listing currently running jobs, detecting transient failures) is lacking.

I've briefly looked into Ververica platform which is an ideal solution except for the licensing aspect. Ideally I'd like to work with something open source. A UI is not necessary.

Has anyone tried ING-Banks open source tool?

Any and all ideas are welcome, thanks!