k8s FileNotFoundException

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

k8s FileNotFoundException

Kaepke, Marc
Hi everyone,

I always get an FileNotFoundException by following the kubernetes setup guide [1].

I moved my jar and my input file onto the job manager pod 
After that I join the job manager pod by using: kubectl exec -it <jobmanager> - - /bin/bash

With ls I can see both files. WordCount-example worked well.

Caused by: java.io.IOException: Error opening the Input Split file:/opt/flink/wiki-vote.txt [0,1095061]: /opt/flink/wiki-vote.txt (No such file or directory)
at org.apache.flink.api.common.io.FileInputFormat.open(FileInputFormat.java:706)
at org.apache.flink.api.common.io.DelimitedInputFormat.open(DelimitedInputFormat.java:477)
at org.apache.flink.api.common.io.GenericCsvInputFormat.open(GenericCsvInputFormat.java:301)
at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:48)
at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:31)
at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:145)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.FileNotFoundException: /opt/flink/wiki-vote.txt (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.flink.core.fs.local.LocalDataInputStream.<init>(LocalDataInputStream.java:49)
at org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:141)
at org.apache.flink.api.common.io.FileInputFormat$InputSplitOpenThread.run(FileInputFormat.java:866)
root@flink-jobmanager-2891032829-pqgwq:/opt/flink# 



Any idea?!


Marc
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: k8s FileNotFoundException

Aljoscha Krettek
Is the file also available on the TaskManager containers? These are doing the actual reading, in fact.

In general, I would suggest to use a distributed file system as input for a Flink Job if you want to process files.

Best,
Aljoscha
On 4. Aug 2017, at 14:13, Kaepke, Marc <[hidden email]> wrote:

Hi everyone,

I always get an FileNotFoundException by following the kubernetes setup guide [1].

I moved my jar and my input file onto the job manager pod 
After that I join the job manager pod by using: kubectl exec -it <jobmanager> - - /bin/bash

With ls I can see both files. WordCount-example worked well.

Caused by: java.io.IOException: Error opening the Input Split file:/opt/flink/wiki-vote.txt [0,1095061]: /opt/flink/wiki-vote.txt (No such file or directory)
at org.apache.flink.api.common.io.FileInputFormat.open(FileInputFormat.java:706)
at org.apache.flink.api.common.io.DelimitedInputFormat.open(DelimitedInputFormat.java:477)
at org.apache.flink.api.common.io.GenericCsvInputFormat.open(GenericCsvInputFormat.java:301)
at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:48)
at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:31)
at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:145)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.FileNotFoundException: /opt/flink/wiki-vote.txt (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.flink.core.fs.local.LocalDataInputStream.<init>(LocalDataInputStream.java:49)
at org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:141)
at org.apache.flink.api.common.io.FileInputFormat$InputSplitOpenThread.run(FileInputFormat.java:866)
root@flink-jobmanager-2891032829-pqgwq:/opt/flink# 



Any idea?!


Marc

Loading...