Data streaming is gaining popularity, as more and more organizations are realizing that the nature of their data production is continuous and unbounded, and can be better served with a streaming architecture. Streaming architectures promise decreased latency from signal to decision, a radically simplified data infrastructure architecture, and the ability to cope with new data that is generated continuously. Apache Flink is a full-featured true stream processing framework with:

• ...

Resource allocation and tuning of large data-parallel pipelines has traditionally been a manual process based on human oversight and as such is costly, wasteful, and high latency. Pipelines might see spikes in input rates, organic traffic growth, or fall behind due to outages or throttling of other services. Typically, such variation forces operators to either overprovision their resources for the worst-case, or to manually monitor and adjust resources when necessary. Both of these...