Weekly Community Update 2019/46

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Weekly Community Update 2019/46

Konstantin Knauf-2
Dear community,

happy to share this week's community update. While slowly approaching the planned feature freeze for Flink 1.10, things have calmed down a bit on the dev mailing list. Stil there are couple of interesting topics to cover. This week's digest includes an update on the Flink 1.8.3 release, a proposal for a KafkaMetricsReporter, a FLIP to improve the configuration of Table API connectors and a bit more.

Flink Development

* [releases] With respect to the release of Flink 1.8.3 the main questions seems to be whether to wait for flink-shaded 9.0. If not, a first release candidate can be expected within the next week. [1]

* [releases] Chesnay proposes to release flink-shaded 9.0 soon and is looking for someone to manage this release. [2]

* [connectors] Stephan has shared a quick update on the progress of FLIP-27 (New Source Interface): a first version will likely be available in 1.10, but it will probably take another release until connectors are migrated and things settle down a bit. [3]

* [connectors, sql] Jark has started a FLIP discussion to improve the properties format of Table API/SQL connectors. It contains a whole list of smaller and larger improvements and it seems this will be targeted for the 1.11 release. [4]

* [metrics] Gyula has started a discussion on contributing a MetricsReporter to write Flink's metrics to Apache Kafka. [5]

* [development process] Dian Fu proposes to introduce a [hidden email] mailing list for users to report security-related issues. There is a lot of positive feedback, but also some concerns,  e.g. because there is already a cross-project [hidden email] mailing list. [6]

Notable Bugs

Not much happening here. So, let's look at two edge cases, which might help one or the other user.

* [FLINK-13184] [1.9.1] [1.8.2] If you are deploying to YARN and start a lot of Taskmanagers (1000s), the Resourcemanager might be blocked/unresponsive quite some time, so that heartbeats of Taskmanagers start timing out. Yang Wang is working on a fix, which might get into 1.8.3. [7]

* [FLINK-14066] If you try to build PyFlink on Windows, this will fail as we use a UNIX specific path to the local Flink distribution's build target. The fix is contained in this ticket. [8]

Events, Blog Posts, Misc

* A missed this excellent Flink Foward Europe recap by my colleagues Robert and Fabian published 1st of November on the Ververica blog in the last weeks, so here it is. [9]

* Upcoming Meetups
    * Bowen Li will speak about "The Rise of Apache Flink and Stream Processing" at the next Big Data Bellevue in Seattle on the 20th of November. [10]
    * The next edition of the Bay Area Apache Flink meetup will happen on the 20th of November with talks by Gyula Fora (Cloudera) and Lakshmi Rao (Lyft).[11]
    * We will have our next Apache Flink Meetup in Munich on November 27th with talks by Heiko Udluft & Giuseppe Sirigu, Airbus, and Konstantin Knauf (on Stateful Functions). [12]
    * There will be an introduction to Apache Flink, use cases and best practices at the next Uber Engineering meeup in Toronto. If you live in Toronto, its an excellent opportunity to get started with Flink or to meet local Flink users. [13]


Konstantin (@snntrable)


Konstantin Knauf | Solutions Architect

+49 160 91394525

Follow us @VervericaData Ververica


Join Flink Forward - The Apache Flink Conference

Stream Processing | Event Driven | Real Time


Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany


Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Tony) Cheng