Using Contextual Data

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Using Contextual Data

Doron Keller

Hello Flink,

 

Is it possible to use contextual data as part of Event Processing (CEP)?

 

I would like to create some tables associated with each key. These tables would need to be updated as information streams in. The values in these tables would also have to be referenced as part of the rule logic. E.g. if a certain value was seen more than x times, trigger.

 

Finally, when the system is starting up these tables would have to be loaded from a database.

 

Thanks,

Doron

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Using Contextual Data

Tzu-Li (Gordon) Tai
Hi,

It seems like you want to load the context tables from external databases when the CEP operators start, and allow pattern detection logic to refer to those tables. I’m not entirely sure if this possible with the Flink CEP library.
I’ve looped in Klou who is more familiar with the CEP library and would probably have more insight on this (its currently the national holidays so we might not be responsive at the moment).

Otherwise, this is definitely achievable using the DataStream API. More specifically, you would want to implement the rich variant functions and load the contextual tables in `open()`. The occurrence frequency of events would be kept as registered state in the case you described.

Hope this helps!

Cheers,
Gordon


On 14 April 2017 at 7:41:46 AM, Doron Keller ([hidden email]) wrote:

Hello Flink,

 

Is it possible to use contextual data as part of Event Processing (CEP)?

 

I would like to create some tables associated with each key. These tables would need to be updated as information streams in. The values in these tables would also have to be referenced as part of the rule logic. E.g. if a certain value was seen more than x times, trigger.

 

Finally, when the system is starting up these tables would have to be loaded from a database.

 

Thanks,

Doron

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Using Contextual Data

Doron Keller
Thank you for your response.

Please let me know if this is doable in CEP, or if you need more information.

Doron


From: "Tzu-Li (Gordon) Tai" <[hidden email]>
Date: Sunday, April 16, 2017 at 11:05 PM
To: "[hidden email]" <[hidden email]>, "[hidden email]" <[hidden email]>
Subject: Re: Using Contextual Data

Hi,

It seems like you want to load the context tables from external databases when the CEP operators start, and allow pattern detection logic to refer to those tables. I’m not entirely sure if this possible with the Flink CEP library.
I’ve looped in Klou who is more familiar with the CEP library and would probably have more insight on this (its currently the national holidays so we might not be responsive at the moment).

Otherwise, this is definitely achievable using the DataStream API. More specifically, you would want to implement the rich variant functions and load the contextual tables in `open()`. The occurrence frequency of events would be kept as registered state in the case you described.

Hope this helps!

Cheers,
Gordon


On 14 April 2017 at 7:41:46 AM, Doron Keller ([hidden email]) wrote:

Hello Flink,

 

Is it possible to use contextual data as part of Event Processing (CEP)?

 

I would like to create some tables associated with each key. These tables would need to be updated as information streams in. The values in these tables would also have to be referenced as part of the rule logic. E.g. if a certain value was seen more than x times, trigger.

 

Finally, when the system is starting up these tables would have to be loaded from a database.

 

Thanks,

Doron

Loading...