Giter Site home page Giter Site logo

Comments (16)

saxiaoju avatar saxiaoju commented on May 30, 2024

大佬，求一下使用方法，alist配置文件是怎么引用的？摸索半天不懂🙃，非常感谢！

from alist-sync.

lee-cq avatar lee-cq commented on May 30, 2024

你好啊，

该项目目前还在开发阶段，还有一些BUG在调试，请谨慎使用。

Github Action 上次测试 OneDrive -> 139pan 速度在30MB/s左右。

后面有时间我会出配置方式（目前我在重构代码）。

非常感谢你对该项目的支持。

from alist-sync.

lee-cq avatar lee-cq commented on May 30, 2024

@saxiaoju 使用RCLone相关流程能简单介绍下吗？

也许在该项目可用前可以封装一个RClone版本的Action，来解决速度问题。

from alist-sync.

ypq123456789 avatar ypq123456789 commented on May 30, 2024

@saxiaoju 使用RCLone相关流程能简单介绍下吗？

也许在该项目可用前可以封装一个RClone版本的Action，来解决速度问题。

老哥好，我使用rclone与咱们这个项目无关哈，是通过reclone挂载alist webdav，然后从onedrive向阿里云盘的加密盘sync，不显示进度，但是看火绒的带宽基本上是很慢的，下载的进度可能只有10兆每秒，上传 也差不多，因为走家宽上限就卡那了

from alist-sync.

user4396 avatar user4396 commented on May 30, 2024

老哥你这咋用？我./bootstarp.sh install 后执行alist-sync copy -h http://localhost:5244 -u admin -p 123456 -s /1 -t /2 提示alist_sync命令未找到啊，咋用？

from alist-sync.

ypq123456789 avatar ypq123456789 commented on May 30, 2024

老哥你这咋用？我./bootstarp.sh install 后执行alist-sync copy -h http://localhost:5244 -u admin -p 123456 -s /1 -t /2 提示alist_sync命令未找到啊，咋用？

等这个老哥出配置方法吧，自己琢磨估计不太行

from alist-sync.

ypq123456789 avatar ypq123456789 commented on May 30, 2024

2a121b22944940bb2db511d9a937292d
用了老哥的配置方法，该配置的都配置了，报错这个

from alist-sync.

lee-cq avatar lee-cq commented on May 30, 2024

2a121b22944940bb2db511d9a937292d 用了老哥的配置方法，该配置的都配置了，报错这个

备份格式不对，ALIST_BACKUP的值应该是在alist网页端，通过备份得到的。

SYNC_CONFIG 和 ALIST_BACKUP 注意区分

from alist-sync.

ypq123456789 avatar ypq123456789 commented on May 30, 2024

2a121b22944940bb2db511d9a937292d 用了老哥的配置方法，该配置的都配置了，报错这个

备份格式不对，ALIST_BACKUP的值应该是在alist网页端，通过备份得到的。

SYNC_CONFIG 和 ALIST_BACKUP 注意区分

我记得我没弄反呀，我为了防止自己记错，重新弄了下这两个，没弄反，还是报这个错误
SYNC_CONFIG内容：

# 如果没有配置MongoDB，文档将会存储至本地缓存
mongodb_uri: "mongodb+srv://${username}:${password}@${host}/alist_sync?retryWrites=true&w=majority&appName=A1"

# 缓存文件夹
cache_dir: ./.alist-sync-cache

# 是否以Daemon模式运行
daemon: false

thread_pool_max_size:
  workers: 5
  scanner: 5
  checker: 5

# Alist 服务器信息 type: list
alist_servers:
  - base_url: http://loaclhost:5244/
    username: "admin"
    password: "123456"
    verify_ssl: false

  - base_url: http://remote_alist_server/
    username: "admin"
    password: "admin"


sync_groups:
  - name: "sync1"
    enable: true  # 默认值: True
    # 同步类型，一共4种：
    # 1 copy:如果目标目录中已经存在该文件，则跳过
    #        忽略存在与目标目录中但不存在于源目录中的文件
    # 2 mirror: 如果目标目录中已经存在该文件，则跳过
    #           删除存在于目录目录但不存在于源目录中的文件
    # 3 sync: 如果目标目录中已经存在该文件，则跳过
    #         删除存在于目录目录但不存在于源目录中的文件
    # 4 sync-incr: 基于文件的修改时间，只同步源目录中修改时间晚于目标目录的文件
    type: "copy"

    # 检查间隔，单位为秒，如果daemon为False，则该值无效
    interval: 300  # 默认值: 300 秒, 5 分钟

    # 是否需要备份，如果为True，则会在同步之前备份目标目录
    # 对于copy，该值无效
    need_backup: false  # 默认值: False

    # 备份目录，当need_backup为True时，该值有效
    # 一个相对目录，最终为每一个group中的每一个server创建一个备份目录
    backup_dir: "./.alist-sync-backup"  # 默认值: ./.alist-sync-backup

    # 黑名单，支持通配符, 使用 fnmatch.fnmatchcase 函数进行匹配
    # 详情参考标准库文档 https://docs.python.org/3/library/fnmatch.html
    # 后面可能会重构，以支持 Linux Glob 模式。
    # 其路径必须相对与Group中定义的目录，或者使用*开头
    # .alist-sync* 将自动添加到黑名单中
    # 例子：
    # 忽略 http://localhost:5244/test1/base/ 目录下的所有目录及文件： "base/*"
    # 忽略所有bfstm文件： "*.bfstm"
    blacklist:
      - "*.bfstm"
      - "base/*"
      - "testa/b/*"

    # 同步目录，一个完整的AList URL，
    # 对于copy, mirror 第一个为源目录，其他个为目标目录
    # Alist服务器信息需要提前在alist_servers中配置
    # 支持在不同的Alist服务器之间同步
    # 例子：http://localhost:5244/test1
    group:
      - "http://loaclhost:5244/OnedriveCF代理"
      - "http://loaclhost:5244/阿里云盘加密"

notify: # 通知服务，当触发一些异常后，将会发送通知。
  - enable: false
    type: email
    smtp_host: ""
    smtp_port: 25
    sender: ""
    password: ""
    recipients:
      - ""

  - enable: false
    type: webhook
    webhook_url: ""
    headers:
      K: V

logs:
  version: 1
  disable_existing_loggers: true
  formatters:
    simple:
      format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
  handlers:
    console:
      class: logging.StreamHandler
      level: DEBUG
      formatter: simple
      stream: ext://sys.stdout

    file:
      class: logging.handlers.RotatingFileHandler
      level: DEBUG
      formatter: simple
      filename: ./logs/alist-sync.log
      maxBytes: 10485760  # 10MB
      backupCount: 5
      encoding: utf8

  loggers:
    alist-sync:
      level: DEBUG
      handlers:
        - console
        - file

    alist-sdk:
      level: INFO
      handlers:
        - console
        - file

ALIST_BACKUP复制的这个txt里面的内容
1709527706022

password和username也设置了
1709527735563

from alist-sync.

lee-cq avatar lee-cq commented on May 30, 2024

  1. 从前面的日志看是JSON解析错误, 我也测试了文件。
    目测，是拷贝内容时没有把文件复制全 (可能是少了最后一个大括号)。
    可以使用Ctrl + A (全选) Ctrl + V （复制）进行拷贝。

  2. ALIST_BACKUP_URL + ALIST_BACKUP_USERNAME + ALIST_BACKUP_PASSWORD 是一个完整的组合.
    这个URL是 公网可访问的Alist Server。 eg: https://al.nn.ci/

from alist-sync.

lee-cq avatar lee-cq commented on May 30, 2024

对于配置文件（SYNC_CONFIG）的建议：

  1. 配置正确的mongodb_uri（可以在mongodb.com上申请一个免费的），否则需要注释这一行，存储在本地方法目前测试不完善，可能会出现问题。
  2. 删除alist_server不使用的项。

from alist-sync.

lee-cq avatar lee-cq commented on May 30, 2024

目前这个版本的传输速度比较慢，后面会继续优化

from alist-sync.

ypq123456789 avatar ypq123456789 commented on May 30, 2024

目前这个版本的传输速度比较慢，后面会继续优化

谢谢老哥，不过我按照你说的改了，现在长这样

# 如果没有配置MongoDB，文档将会存储至本地缓存
mongodb_uri: "mongodb+srv://648558021:[email protected]/?retryWrites=true&w=majority&appName=Cluster0"

# 缓存文件夹
cache_dir: ./.alist-sync-cache

# 是否以Daemon模式运行
daemon: false

thread_pool_max_size:
  workers: 5
  scanner: 5
  checker: 5

# Alist 服务器信息 type: list
alist_servers:
  - base_url: https://alist.bacon123.eu.org
    username: "上面这个网址的账号"
    password: "上面这个网址的密码"

sync_groups:
  - name: "sync1"
    enable: true  # 默认值: True
    # 同步类型，一共4种：
    # 1 copy:如果目标目录中已经存在该文件，则跳过
    #        忽略存在与目标目录中但不存在于源目录中的文件
    # 2 mirror: 如果目标目录中已经存在该文件，则跳过
    #           删除存在于目录目录但不存在于源目录中的文件
    # 3 sync: 如果目标目录中已经存在该文件，则跳过
    #         删除存在于目录目录但不存在于源目录中的文件
    # 4 sync-incr: 基于文件的修改时间，只同步源目录中修改时间晚于目标目录的文件
    type: "copy"

    # 检查间隔，单位为秒，如果daemon为False，则该值无效
    interval: 300  # 默认值: 300 秒, 5 分钟

    # 是否需要备份，如果为True，则会在同步之前备份目标目录
    # 对于copy，该值无效
    need_backup: false  # 默认值: False

    # 备份目录，当need_backup为True时，该值有效
    # 一个相对目录，最终为每一个group中的每一个server创建一个备份目录
    backup_dir: "./.alist-sync-backup"  # 默认值: ./.alist-sync-backup

    # 黑名单，支持通配符, 使用 fnmatch.fnmatchcase 函数进行匹配
    # 详情参考标准库文档 https://docs.python.org/3/library/fnmatch.html
    # 后面可能会重构，以支持 Linux Glob 模式。
    # 其路径必须相对与Group中定义的目录，或者使用*开头
    # .alist-sync* 将自动添加到黑名单中
    # 例子：
    # 忽略 http://localhost:5244/test1/base/ 目录下的所有目录及文件： "base/*"
    # 忽略所有bfstm文件： "*.bfstm"
    blacklist:
      - "*.bfstm"
      - "base/*"
      - "testa/b/*"

    # 同步目录，一个完整的AList URL，
    # 对于copy, mirror 第一个为源目录，其他个为目标目录
    # Alist服务器信息需要提前在alist_servers中配置
    # 支持在不同的Alist服务器之间同步
    # 例子：http://localhost:5244/test1
    group:
      - "https://alist.bacon123.eu.org/OnedriveCF"
      - "https://alist.bacon123.eu.org/Alicrypt"

notify: # 通知服务，当触发一些异常后，将会发送通知。
  - enable: false
    type: email
    smtp_host: ""
    smtp_port: 25
    sender: ""
    password: ""
    recipients:
      - ""

  - enable: false
    type: webhook
    webhook_url: ""
    headers:
      K: V

logs:
  version: 1
  disable_existing_loggers: true
  formatters:
    simple:
      format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
  handlers:
    console:
      class: logging.StreamHandler
      level: DEBUG
      formatter: simple
      stream: ext://sys.stdout

    file:
      class: logging.handlers.RotatingFileHandler
      level: DEBUG
      formatter: simple
      filename: ./logs/alist-sync.log
      maxBytes: 10485760  # 10MB
      backupCount: 5
      encoding: utf8

  loggers:
    alist-sync:
      level: DEBUG
      handlers:
        - console
        - file

    alist-sdk:
      level: INFO
      handlers:
        - console
        - file

然后，又出现新的错误了：

Run # 这将会导入全部的内容包括：设置，元数据，用户，存储器。
  # 这将会导入全部的内容包括：设置，元数据，用户，存储器。
  echo $(pwd)
  cat > alist-backup-config.json << EOF
  
  EOF
  python3 tools/create_storage.py
  shell: /usr/bin/bash -e ***0***
  env:
    _ALIST_SYNC_NAME: action-***789-8169031397-[2](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:2)4
    _ALIST_SYNC_DEBUG: false
    _ALIST_ADMIN_PASSWORD: ***
    pythonLocation: /opt/hostedtoolcache/Python/[3](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:3).11.8/x6[4](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:4)
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.11.8/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.8/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.8/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.8/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.8/x64/lib
    _ALIST_BACKUP: ***
    _ALIST_BACKUP_URL: ***
    _ALIST_BACKUP_USERNAME: ***
    _ALIST_BACKUP_PASSWORD: ***
    _RELOAD_STORAGE: true
/home/runner/work/alist-sync/alist-sync
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions
local_client = ***
    yield
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpx/_transports/default.py", line 233, in handle_request
    resp = self._pool.handle_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request
    raise exc from None
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request
    response = connection.handle_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 101, in handle_request
    return self._connection.handle_request(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 143, in handle_request
    raise exc
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 113, in handle_request
    ) = self._receive_response_headers(**kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 186, in _receive_response_headers
    event = self._receive_event(timeout=timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpcore/_sync/http11.py", line 238, in _receive_event
    raise RemoteProtocolError(msg)
httpcore.RemoteProtocolError: Server disconnected without sending a response.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/home/runner/work/alist-sync/alist-sync/tools/create_storage.py", line 77, in <module>
    local_client.import_config_from_other_client(
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/alist_sdk/tools/client.py", line 2[5](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:5), in import_config_from_other_client
    other_client = self.__class__(
                   ^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x[6](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:6)4/lib/python3.11/site-packages/alist_sdk/client.py", line 45, in __init__
    self.login(username, password, has_opt)
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/alist_sdk/client.py", line 126, in login
    res = self.post(
          ^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpx/_client.py", line 1145, in post
    return self.request(
           ^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/alist_sdk/client.py", line 89, in request
    return super().request(method, url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpx/_client.py", line 82[7](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:7), in request
_RELOAD_STORAGE = true
code=200 message='success' data=ID(id=1)
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.[8](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:8)/x64/lib/python3.11/site-packages/httpx/_client.py", line [9](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:9)14, in send
    response = self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpx/_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpx/_client.py", line 979, in _send_handling_redirects
    response = self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/httpx/_client.py", line [10](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:10)15, in _send_single_request
    response = transport.handle_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.[11](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:11).8/x64/lib/python3.11/site-packages/httpx/_transports/default.py", line 232, in handle_request
    with map_httpcore_exceptions():
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/contextlib.py", line [15](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:15)8, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/opt/hostedtoolcache/Python/3.11.8/x[64](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:65)/lib/python3.11/site-packages/httpx/_transports/default.py", line [86](https://github.com/ypq123456789/alist-sync/actions/runs/8169031397/job/22332271262#step:6:87), in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.RemoteProtocolError: Server disconnected without sending a response.
Error: Process completed with exit code 1.

from alist-sync.

lee-cq avatar lee-cq commented on May 30, 2024

貌似是无法连接到请求远程服务器，如果源站在CF后面，可能被CF拦截了。

from alist-sync.

ypq123456789 avatar ypq123456789 commented on May 30, 2024

貌似是无法连接到请求远程服务器，如果源站在CF后面，可能被CF拦截了。

谢谢老哥回复，我这个网址确实是解析在cf的，但是我没套cf的cdn啊，安全设置基本上都是关的，这种能通过cf的设置解决嘛
image
image
image
image
image
image

from alist-sync.

ypq123456789 avatar ypq123456789 commented on May 30, 2024

大佬还更新吗

from alist-sync.

Related Issues (9)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.