Windows version can't create a job due to credentialsFilePath

[redis-field-engineering]

Windows version can't create a job due to credentialsFilePath

[viragtripathi]

Hi @ronan-vargas, I don't see your Redis (target) credentials file (or maybe it's not visible in the screenshot you have provided). Can you please try this,

Move all credentials file from the location you have chosen for credentialsFilePath except these,

• redisconnect_credentials_jobmanager.properties (this is used for the metadata DB connection to store all of Redis Connect metadata)
• redisconnect_credentials_redis_cdc-job.properties (this is used for the target Redis DB connection)
• redisconnect_credentials_oracle_cdc-job.properties (this is used for the source i.e. Oracle connection)

The file format for Redis Connect credentials are as follows,
redisconnect_credentials_<source|target>_.properties, the source name is same as database product name on the jdbc url and target is always redis. I noticed you have one file with UPPERCASE ORACLE and that shouldn't be there either.

redisconnect_credentials_jobmanager.properties, there will always be one jobmanager file per Redis Connect instance/node.

The fact that you're able to start Redis Connect instance means you had no issue connecting to your metadata Redis DB and the credentialsPath for this must be correct in the jobmanager.properties file which is read at the Instance startup to connect to Redis so all of Redis Connect metadata can be persisted in Redis. Also, your Redis Enterprise DB endpoint doesn't seem correct so please fix the Redis URI and provide a valid Redis Enterprise DB endpoint.

[ronan-vargas]

Hello @viragtripathi, thank you very much for answering me.

After reviewing some steps (before your answer) I've successfully created a job through Redis Connect. For to do that, I also needed to put "ojdbc*.jar" in the "lib" directory and also used "cdc-job" as jobName, instead of "cdc_job".
However, now Redis Connect throws an exception when I try to start the job for the first time, using 'load' as jobType.

2022-07-26 10:42:32,773 ERROR redis-connect-manager [JobManagerThreadpool-2] Instance: 22996@IBM-PF3R4N79 failed to initialize JobId: {connect}:task:partition:2:cdc-job:C##RCUSER.EMP EXCEPTION MESSAGE: RedisConnectValidationException: STACKTRACE: [Validation Failure: InstanceId: 22996@IBM-PF3R4N79 will not start JobId: {connect}:task:partition:2:cdc-job:C##RCUSER.EMP because there are no records in Table: C##RCUSER.EMP, at com.redis.connect.pipeline.producer.impl.load.RDBInitialLoadProducer.startSourceConnection(RDBInitialLoadProducer.java:78), at com.redis.connect.cluster.service.JobManager.startJobProducer(JobManager.java:713), at com.redis.connect.cluster.service.JobClaimer.initializeJob(JobClaimer.java:214), at com.redis.connect.cluster.service.JobClaimer.claimJob(JobClaimer.java:171), at com.redis.connect.cluster.service.JobClaimer.run(JobClaimer.java:81), at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539), at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305), at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305), at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136), at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635), at java.base/java.lang.Thread.run(Thread.java:833)]

The problem is that my C##RCUSER.EMP table has records and these records was inserted before starting the job.

select count(*) from C##RCUSER.emp;

count(*)

11

My configuration files and Connect logs are in attachment:

cdc-job.json.txt
redisconnect_credentials_jobmanager.properties.txt
redisconnect_credentials_oracle_cdc-job.properties.txt
redisconnect_credentials_redis_cdc-job.properties.txt
redis-connect-22996.log.txt
redis-connect-manager-22996.log.txt

[viragtripathi]

Thanks @ronan-vargas you already made good progress. You're right for Oracle, you do need the client driver jar as we can't include it due to licensing. I see you have some issue with the driver you're using. From the connector log, No suitable driver STACKTRACE: [java.sql.SQLException: No suitable driver

Can you try this Oracle JDBC Driver and restart your instance then re-run the oracle-job? Also, please change the partition to 1 if you only have 11 records to load otherwise you would fail in another validation as <500 records can only have 1 partition OR if you want to add more records then see here, https://github.com/redis-field-engineering/redis-connect-dist/tree/main/examples/oracle/demo#initial-loader-steps

[ronan-vargas]

Hello,
Considering initial loader steps link, I've used a distinct Oracle docker image, and this one doesn't have "sqlldr" tool. Consequently, I've inserted records on "emp" table directly by Oracle SQL Developer.
Moreover, I've noticed that "load_sql.sh" script references a table called "employees". However, I think that previous script only creates "emp" table.

About your last answer, even changing "partitions" property in .json configuration file, the error "there are no records in Table: C##RCUSER.EMP" is still occurring.

Couldn't be the "#" in user name (C##RCUSER) a problem? My point is that "#" character is commonly used for comments in Properties files.

[viragtripathi]

Please see, https://github.com/redis-field-engineering/redis-connect-dist/blob/main/examples/oracle/demo/load_sql.sh#L26 for the load if you still need it.

The error was due to the driver you're using earlier. Have you replaced the driver with the link I sent?

[ronan-vargas]

The driver that I was using is also called "ojdbc8.jar", but the file in my machine was a bit larger (4963 KiB vs. 4369 KiB).
I've replaced the file with the one you suggested, restarted Redis Connect, updated the job again and tried to start.
Unfortunately, the exception "there are no records in Table..." is still occurring.
Now I'll try this new configuration (I'm using Oracle 12c) for *JAVA_OPTIONS and I'll send a message in case of any problem.

Thanks in advance.

[viragtripathi]

If you're using 12c then you shouldn't see Got minus one from a read call error I mentioned earlier. Are you still seeing any driver related error in the connector log i.e. redis-connect-<PID>.log?

[viragtripathi]

Hi @ronan-vargas as discussed on the call, I am going to close this issue as all of your setup issue has been resolved and you have both load and stream job running with Oracle as a source and Redis Enterprise DB as target.

[ronan-vargas]

Hi @viragtripathi , OK, but the only thing that isn`t so clear to me is the fact that the setup works in "Initial Load" scenario (even with the Oracle's service name with ".localdomain" suffix) but it doesn't work in "CDC" scenario.
I'll send by email the contact of everyone that was in the meeting.

[viragtripathi]

Hi @ronan-vargas, it probably works with initial load as it's a different connection using the databaseURL in the payload but stream job i.e. cdc uses a different connection that is started and managed by Debezium based on the database. properties and when you pass database.dbname with .localdomain suffix, you see the oracle driver error with TNS listener issue. This is why your Oracle developer connection also worked with .localdomain as it's also using the same JDBC URL like the initial loader.