Theta join with CrossFunction

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Theta join with CrossFunction

Arvid Heise
Hi Flinker,

how can you implement a theta join with the revised cross API? In particular, how can you _not_ emit for a given input pair?

With the old interface, not calling the collector was quite easy and obvious. Returning null in the new interface seems to be a bad idea.

Or is there a different way to implement a theta join with the current Flink API? FlatMap with Broadcast?

Best,

Arvid
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Theta join with CrossFunction

Stephan Ewen
Hi!

You can always use a cross without a function (that creates a Tuple2 of the input) and then use Filter on the result, or flatMap, or anything that implements your predicate.


Stephan


Loading...