“Hope that helps”…
I have dug through the javacpp source code to find how the Loader class uses the temp cache location for the native libraries and in my open method in my RichMapFunction I am now setting the System.property to a random location so if the job restarts and calls open again, it uses a random temp location as in your example.
So thank you so much, got me on the right path. Now I’ve got a problem with out of memory errors arrghh (I will start another thread on this as I don’t want to soil this one with a different topic)
I'm afraid you're running in the same problem that we already encountered with loading the native RocksDB library: https://github.com/apache/flink/blob/219ae33d36e67e3e74f493cf4956a290bc966a5d/flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBStateBackend.java#L519
The code section has a good description of what is going on. We're using NativeLibraryLoader , which comes with RocksDB to try and load the native library from a different temporary location if loading it the normal way fails. (The relocation of the lib to a temp dir is in the NativeLibraryLoader, not on our side. We're just providing a temp path for NativeLibraryLoader to work with.)
Hope that helps,
***This email originated outside SecureData***
Click <a href="https://www.mailcontrol.com/sr/sqqAKS4fcuTGX2PQPOmvUsrLibhXE7+S2XLa08iemTAEqT34+e6paNr2uTugfjZ5BFH3GVoNO4qqE4naC2k80A=="> here to report this email as spam.
|Free forum by Nabble||Edit this page|