Problems with pikpak storing hash after uploads

[djniche]

Problems with pikpak storing hash after uploads

[djniche]

Add: only occasionally a few files will appear this problem, not 100%, when I have been retained pikpak source files, do not delete the case, after a period of time will be changed once the hash value, this time the value is correct. After deleting the file I can restore the file by the hash value.
Files with changing hash values can also be played normally, provided they are not deleted from pikpak.

PikPak://MEYD-868-4K.mp4|24055707717|3927AE19823CE8E020402C4C7650CD8FD37E14D1 The hash of the error file that I synchronized with rclone before
PikPak://MEYD-868-4K.mp4|24055707717|6C97D756D7622262EA75F6257738D6E39FF5368E This is the hash I generated after copying pikpak via alist googledrive. The former is invalidated by the deletion of the source file.

[wiserain]

At the time of writing pikpak for rclone, hash used for uploading and verifying file existence wasn't clearly discovered. So I just let it a simple sha1. However, I recently found that it is sha1 of sha1 of file blocks which is similar with torrent piece hash.

Once another fix #7787 which is closely related to this one is sorted out, I will make a new PR for it. Thanks.

[djniche]

At the time of writing pikpak for rclone, hash used for uploading and verifying file existence wasn't clearly discovered. So I just let it a simple sha1. However, I recently found that it is sha1 of sha1 of file blocks which is similar with torrent piece hash.

Once another fix #7787 which is closely related to this one is sorted out, I will make a new PR for it. Thanks.

I know, it's named gcid. The algorithm is written in some Chinese article
Where xunlei is pikpak's algorithm, because pikpak bought the back-end architecture from xunlei, so the calculation method is the same.
https://binux.blog/2012/03/hash_algorithm_of_xunlei/
https://post.smzdm.com/p/awzm3o3m/
(alist project contains pikpak, alist is an open source project , you can look at the specific implementation of the code .
https://github.com/alist-org/alist/tree/main/drivers/pikpak)
def gcid_hash_file(path):
h = hashlib.sha1()
size = os.path.getsize(path)
psize = 0x40000
while size / psize > 0x200 and psize < 0x200000:
psize = psize << 1
with open(path, 'rb') as stream:
data = stream.read(psize)
while data:
h.update(hashlib.sha1(data).digest())
data = stream.read(psize)
return h.hexdigest().upper()

[wiserain]

Thanks for the useful link. Alist is my source of information as well :)