Use processing time and time window flink acts like batch mode.

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Use processing time and time window flink acts like batch mode.

yunfan123
I using processing time and the data source comes from kafka.
My code is like follows:
    streams.keyBy(XXX)
               .timeWindow(Time.seconds(30))
               .apply(myClassObject)

Log in myClassObject is like:
2017-07-12 20:00:00,
2017-07-12 20:00:00,
........
2017-07-12 20:00:30,
2017-07-12 20:00:30,
...............
2017-07-12 20:01:00,
2017-07-12 20:01:00,

It means every 30 seconds, all of the window be triggered once.
And I'm sure the amounts of my key is very large.
So in the program based my time window length is 30, my expects is  :
key1: processing time 1
key2: processing time 1
key3: processing time 2
key4: processing time 3
The key1 and key2 should be processed in time 31.
The key3 should be processed in time 32.
The key4 should be processed in time 33.



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Use processing time and time window flink acts like batch mode.

Aljoscha Krettek
Hi,

Could you please clarify what is the expected and the actual output? Also, could you maybe post the snipped that produces the log output?

Best,
Aljoscha

> On 12. Jul 2017, at 15:32, yunfan123 <[hidden email]> wrote:
>
> I using processing time and the data source comes from kafka.
> My code is like follows:
>    streams.keyBy(XXX)
>               .timeWindow(Time.seconds(30))
>               .apply(myClassObject)
>
> Log in myClassObject is like:
> 2017-07-12 20:00:00,
> 2017-07-12 20:00:00,
> ........
> 2017-07-12 20:00:30,
> 2017-07-12 20:00:30,
> ...............
> 2017-07-12 20:01:00,
> 2017-07-12 20:01:00,
>
> It means every 30 seconds, all of the window be triggered once.
> And I'm sure the amounts of my key is very large.
> So in the program based my time window length is 30, my expects is  :
> key1: processing time 1
> key2: processing time 1
> key3: processing time 2
> key4: processing time 3
> The key1 and key2 should be processed in time 31.
> The key3 should be processed in time 32.
> The key4 should be processed in time 33.
>
>
>
>
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Use-processing-time-and-time-window-flink-acts-like-batch-mode-tp14211.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Loading...