rest.port is reset to 0 by YarnEntrypointUtils

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

rest.port is reset to 0 by YarnEntrypointUtils

김동원
Hi,

I'm trying to launch a dispatcher on top of YARN by executing "yarn-session.sh" on the command line.

To access the rest endpoint outside the cluster, I need to assign a port from an allowed range.

YarnEntrypointUtils, however, sets rest.port to 0 for random binding.

Is there any reason on it?


Best,

- Dongwon
Reply | Threaded
Open this post in threaded view
|

Re: rest.port is reset to 0 by YarnEntrypointUtils

Gary Yao-2
Hi Dongwon,

I think the rationale was to avoid conflicts between multiple Flink instances
running on the same YARN cluster. There is a ticket that proposes to allow
configuring a port range instead [1].
 
Best,
Gary

[1] https://issues.apache.org/jira/browse/FLINK-5758

On Tue, Apr 17, 2018 at 9:56 AM, Dongwon Kim <[hidden email]> wrote:
Hi,

I'm trying to launch a dispatcher on top of YARN by executing "yarn-session.sh" on the command line.

To access the rest endpoint outside the cluster, I need to assign a port from an allowed range.

YarnEntrypointUtils, however, sets rest.port to 0 for random binding.

Is there any reason on it?


Best,

- Dongwon

Reply | Threaded
Open this post in threaded view
|

Re: rest.port is reset to 0 by YarnEntrypointUtils

김동원
Hi Gary,

Thanks a lot for replay.
Hope the issue is resolved soon.

I have a suggestion regarding the rest port.
Considering the role of dispatcher, it needs to have its own port range that is not shared by job managers spawned by dispatcher.
If I understand FLIP-6 correctly, only a few dispatchers are going to be launched as a kind of gateway and they should be visible outside the cluster.
it seems quite reasonable to give them a range of ports that are only consumed by dispatcher.

Best,

- Dongwon



2018. 4. 18. 오후 6:40, Gary Yao <[hidden email]> 작성:

Hi Dongwon,

I think the rationale was to avoid conflicts between multiple Flink instances
running on the same YARN cluster. There is a ticket that proposes to allow
configuring a port range instead [1].
 
Best,
Gary

[1] https://issues.apache.org/jira/browse/FLINK-5758

On Tue, Apr 17, 2018 at 9:56 AM, Dongwon Kim <[hidden email]> wrote:
Hi,

I'm trying to launch a dispatcher on top of YARN by executing "yarn-session.sh" on the command line.

To access the rest endpoint outside the cluster, I need to assign a port from an allowed range.

YarnEntrypointUtils, however, sets rest.port to 0 for random binding.

Is there any reason on it?


Best,

- Dongwon