Using DataManage4BigData class

[bjfu-ai-institute]

Using DataManage4BigData class

[vzxxbacq]

Hi, @pranoot .

The 'split_type' is used for storing different part of your data like train, test, and they will be stored in save_path/data/split_type.
And I'm sorry I didn't apply the method to handle this situation, but it's not difficult to implement. I can rewrite write_file in several days.

[pranoot]

Hey Thank you for the revert. Ok I'll write the script and try to implement it. Just wanted to check if I was doing anything wrong.

[vzxxbacq]

Hello @pranoot , I changed the code that allow us to write dataset in several steps, now we can write a script to save different parts of dataset the datamanger will be auto reloaded ( the path in the config should be the same. ), and you can also use clear() method to delete saved file. A simple example in the deep_speaker.py.

[vzxxbacq]

If you still have questions, feel free to reopen it~

[pranoot]

Hii,
I am running the script and its giving me this error.

RuntimeWarning: invalid value encountered in float_scalars
  return np.dot(vector1, vector2) / (np.linalg.norm(vector1) * (np.linalg.norm(vector2)))

Can you please help me withthe same?
The error is in the cosine function.

Thank you !