Data streaming is gaining popularity, as more and more organizations are realizing that the nature of their data production is continuous and unbounded, and can be better served with a streaming architecture. Streaming architectures promise decreased latency from signal to decision, a radically simplified data infrastructure architecture, and the ability to cope with new data that is generated continuously. Apache Flink is a full-featured true stream processing framework with:

• Easy to use Java- and Scala-embedded APIs that make stream analytics easy, yet provide powerful tools to deal with time and uncertainty
• Throughput close to a million of events per second per core
• Latencies as low as the millisecond range
• Full support for event time and out of order arrivals with flexible windows, watermarks, and triggers
• Exactly-once consistency guarantees, and the ability to realize distributed transactional data movement between systems (e.g., between Kafka and HDFS)
• Ease of configuration and separation between application logic and fault tolerance via a novel asynchronous checkpointing algorithm
• No single point of failure
• Integration with popular open source infrastructure (e.g., Hadoop, HBase, Kafka, Cascading, Elasticsearch, …)
• Batch processing as a special case of stream processing, including dedicated libraries for machine learning and graph processing, managed memory on-, and off-heap, and query optimization

Flink is used in several companies, including at ResearchGate, Bouygues Telecom, the Otto Group, and Capital One, and has a large and active developer community of well over 140 contributors. In this talk, we provide an overview of the system internals and its streaming-first philosophy, as well as the programming APIs.