Does apache flink support stream input from Postgresql ?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Does apache flink support stream input from Postgresql ?

Yu Watanabe
Hello .

I would like to ask question about possibility of stream read table rows  from PostgresQL using pyflink.

In use cases in online document, there is a example which is reading data from transactional database.


Is it possible to stream read from PostgresQL  using pyflink ?

Thanks,
Yu Watanabe

--
Yu Watanabe
Weekend Freelancer who loves to challenge building data platform
[hidden email]
LinkedIn icon  Twitter icon  
Reply | Threaded
Open this post in threaded view
|

Re: Does apache flink support stream input from Postgresql ?

Zhenghua Gao
The jdbc connector can read data from PostgreSQL for Table/SQL users.
For pyflink, cc @Hequn

Best Regards,
Zhenghua Gao


On Wed, Nov 20, 2019 at 7:56 PM Yu Watanabe <[hidden email]> wrote:
Hello .

I would like to ask question about possibility of stream read table rows  from PostgresQL using pyflink.

In use cases in online document, there is a example which is reading data from transactional database.


Is it possible to stream read from PostgresQL  using pyflink ?

Thanks,
Yu Watanabe

--
Yu Watanabe
Weekend Freelancer who loves to challenge building data platform
[hidden email]
LinkedIn icon  Twitter icon  
Reply | Threaded
Open this post in threaded view
|

Re: Does apache flink support stream input from Postgresql ?

Theo Diefenthal
In reply to this post by Yu Watanabe
Hi,

You could check out Debezium [1] and get your database CDC into a kafka stream which can then be read nicely by flink.

Best regards
Theo

[1] https://debezium.io/documentation/reference/1.0/tutorial.html


Von: "Yu Watanabe" <[hidden email]>
An: "user" <[hidden email]>
Gesendet: Mittwoch, 20. November 2019 12:56:18
Betreff: Does apache flink support stream input from Postgresql ?

Hello .

I would like to ask question about possibility of stream read table rows  from PostgresQL using pyflink.

In use cases in online document, there is a example which is reading data from transactional database.


Is it possible to stream read from PostgresQL  using pyflink ?

Thanks,
Yu Watanabe

--
Yu Watanabe
Weekend Freelancer who loves to challenge building data platform
[hidden email]
LinkedIn icon  Twitter icon  


--
SCOOP Software GmbH - Gut Maarhausen - Eiler Straße 3 P - D-51107 Köln
Theo Diefenthal

T +49 221 801916-196 - F +49 221 801916-17 - M +49 160 90506575
[hidden email] - www.scoop-software.de
Sitz der Gesellschaft: Köln, Handelsregister: Köln,
Handelsregisternummer: HRB 36625
Geschäftsführung: Dr. Oleg Balovnev, Frank Heinen,
Martin Müller-Rohde, Dr. Wolfgang Reddig, Roland Scheel