AI-First Process Automation with Large ([Language (LLMs) / Action (LAMs) / Multimodal (LMMs)] / Visual Language (VLMs)) Models
Home Page: https://www.OpenAdapt.AI
License: MIT License
How can we enable patients and/or patient advocates to automate their workflows?
This task involves creating recordings involving one or more relevant applications, and implementing tests for corresponding completions:
How can we use RWKV-LMM to implement "infinite" context lengths?
From https://github.com/BlinkDL/RWKV-LM:
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
Additional reading:
Hyena Hierarchy: Towards Larger Convolutional Language Models
Whenever I try to play back a recording, I get this error. I'm not sure what the playbacks are supposed to look like since I can't test them. Any help is appreciated!
Traceback (most recent call last):
File "C:\Users\Angel\Desktop\puterbot\puterbot\replay.py", line 43, in <module>
fire.Fire(replay)
File "C:\Users\Angel\Desktop\puterbot\venv\lib\site-packages\fire\core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "C:\Users\Angel\Desktop\puterbot\venv\lib\site-packages\fire\core.py", line 466, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "C:\Users\Angel\Desktop\puterbot\venv\lib\site-packages\fire\core.py", line 681, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "C:\Users\Angel\Desktop\puterbot\puterbot\replay.py", line 21, in replay
strategy_class_by_name = get_strategy_class_by_name()
File "C:\Users\Angel\Desktop\puterbot\puterbot\utils.py", line 375, in get_strategy_class_by_name
from strategies import BaseReplayStrategy
File "C:\Users\Angel\Desktop\puterbot\puterbot\strategies\__init__.py", line 1, in <module>
from strategies.base import BaseReplayStrategy
File "C:\Users\Angel\Desktop\puterbot\puterbot\strategies\base.py", line 15, in <module>
from puterbot.models import InputEvent, Recording, Screenshot
File "C:\Users\Angel\Desktop\puterbot\../puterbot\puterbot\models.py", line 13, in <module>
class Recording(Base):
File "C:\Users\Angel\Desktop\puterbot\venv\lib\site-packages\sqlalchemy\orm\decl_api.py", line 76, in __init__
_as_declarative(reg, cls, dict_)
File "C:\Users\Angel\Desktop\puterbot\venv\lib\site-packages\sqlalchemy\orm\decl_base.py", line 126, in _as_declarative
return _MapperConfig.setup_mapping(registry, cls, dict_, None, {})
File "C:\Users\Angel\Desktop\puterbot\venv\lib\site-packages\sqlalchemy\orm\decl_base.py", line 183, in setup_mapping
return cfg_cls(registry, cls_, dict_, table, mapper_kw)
File "C:\Users\Angel\Desktop\puterbot\venv\lib\site-packages\sqlalchemy\orm\decl_base.py", line 331, in __init__
self._setup_table(table)
File "C:\Users\Angel\Desktop\puterbot\venv\lib\site-packages\sqlalchemy\orm\decl_base.py", line 854, in _setup_table
table_cls(
File "<string>", line 2, in __new__
File "C:\Users\Angel\Desktop\puterbot\venv\lib\site-packages\sqlalchemy\util\deprecations.py", line 309, in warned
return fn(*args, **kwargs)
File "C:\Users\Angel\Desktop\puterbot\venv\lib\site-packages\sqlalchemy\sql\schema.py", line 594, in __new__
raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Table 'recording' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.
How can we enable SDRs to automate their workflows?
This task involves creating recordings involving one or more relevant applications, and implementing tests for corresponding completions:
2023-04-30 18:57:43.832 | INFO | __mp_main__:write_events:230 - event_type='screen' starting
2023-04-30 18:57:43.832 | INFO | __mp_main__:write_events:230 - event_type='window' starting
2023-04-30 18:57:43.833 | INFO | __mp_main__:write_events:230 - event_type='input' starting
^C2023-04-30 19:00:57.042 | INFO | __mp_main__:write_events:239 - event_type='window' done
2023-04-30 19:00:57.042 | INFO | __main__:record:679 - joining...
2023-04-30 19:00:57.043 | INFO | __main__:read_screen_events:367 - done
2023-04-30 19:00:57.043 | INFO | __mp_main__:write_events:239 - event_type='input' done
2023-04-30 19:00:57.044 | INFO | __main__:process_events:145 - done
<CTRL+C>:
2023-04-30 19:02:45.690 | INFO | __mp_main__:write_events:239 - event_type='screen' done
^CTraceback (most recent call last):
File "/usr/local/Cellar/[email protected]/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/Cellar/[email protected]/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/Users/abrichr/MLDSAI/src/puterbot/puterbot/record.py", line 695, in <module>
fire.Fire(record)
File "/Users/abrichr/MLDSAI/src/puterbot/.venv/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/Users/abrichr/MLDSAI/src/puterbot/.venv/lib/python3.10/site-packages/fire/core.py", line 466, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/Users/abrichr/MLDSAI/src/puterbot/.venv/lib/python3.10/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/Users/abrichr/MLDSAI/src/puterbot/puterbot/record.py", line 685, in record
screen_event_writer.join()
File "/usr/local/Cellar/[email protected]/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 149, in join
res = self._popen.wait(timeout)
File "/usr/local/Cellar/[email protected]/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_fork.py", line 43, in wait
return self.poll(os.WNOHANG if timeout == 0.0 else 0)
File "/usr/local/Cellar/[email protected]/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_fork.py", line 27, in poll
pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt
^CException ignored in atexit callback: <function _exit_function at 0x13479b6d0>
Traceback (most recent call last):
File "/usr/local/Cellar/[email protected]/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/util.py", line 357, in _exit_function
p.join()
File "/usr/local/Cellar/[email protected]/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 149, in join
res = self._popen.wait(timeout)
File "/usr/local/Cellar/[email protected]/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_fork.py", line 43, in wait
return self.poll(os.WNOHANG if timeout == 0.0 else 0)
File "/usr/local/Cellar/[email protected]/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_fork.py", line 27, in poll
pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt:
There are several issues involving memory consumption.
This task involves defining the bottlenecks and understanding tradeoffs between available options.
.env to configure programmatically e.g. via terraform and/or Python wrapped aws clihttps://platform.openai.com/docs/plugins/getting-started
The minimal definition of the required ai-plugin.json file will look like the following:
{
"schema_version": "v1",
"name_for_human": "TODO Plugin",
"name_for_model": "todo",
"description_for_human": "Plugin for managing a TODO list. You can add, remove and view your TODOs.",
"description_for_model": "Plugin for managing a TODO list. You can add, remove and view your TODOs.",
"auth": {
"type": "none"
},
"api": {
"type": "openapi",
"url": "http://localhost:3333/openapi.yaml",
"is_user_authenticated": false
},
"logo_url": "http://localhost:3333/logo.png",
"contact_email": "[email protected]",
"legal_info_url": "http://www.example.com/legal"
}
The OpenAPI specification is the wrapper that sits on top of your API. A basic OpenAPI specification will look like the following:
openapi: 3.0.1
info:
title: TODO Plugin
description: A plugin that allows the user to create and manage a TODO list using ChatGPT.
version: 'v1'
servers:
- url: http://localhost:3333
paths:
/todos:
get:
operationId: getTodos
summary: Get the list of todos
responses:
"200":
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/getTodosResponse'
components:
schemas:
getTodosResponse:
type: object
properties:
todos:
type: array
items:
type: string
description: The list of todos.
The following Python code is an example of how you can set up a simple proxy of your public facing API.
import requests
import os
import yaml
from flask import Flask, jsonify, Response, request, send_from_directory
from flask_cors import CORS
app = Flask(__name__)
PORT = 3333
# Note: Setting CORS to allow chat.openapi.com is required for ChatGPT to access your plugin
CORS(app, origins=[f"http://localhost:{PORT}", "https://chat.openai.com"])
api_url = 'https://example.com'
@app.route('/.well-known/ai-plugin.json')
def serve_manifest():
return send_from_directory(os.path.dirname(__file__), 'ai-plugin.json')
@app.route('/openapi.yaml')
def serve_openapi_yaml():
with open(os.path.join(os.path.dirname(__file__), 'openapi.yaml'), 'r') as f:
yaml_data = f.read()
yaml_data = yaml.load(yaml_data, Loader=yaml.FullLoader)
return jsonify(yaml_data)
@app.route('/openapi.json')
def serve_openapi_json():
return send_from_directory(os.path.dirname(__file__), 'openapi.json')
@app.route('/<path:path>', methods=['GET', 'POST'])
def wrapper(path):
headers = {
'Content-Type': 'application/json',
}
url = f'{api_url}/{path}'
print(f'Forwarding call: {request.method} {path} -> {url}')
if request.method == 'GET':
response = requests.get(url, headers=headers, params=request.args)
elif request.method == 'POST':
print(request.headers)
response = requests.post(url, headers=headers, params=request.args, json=request.json)
else:
raise NotImplementedError(f'Method {request.method} not implemented in wrapper for {path=}')
return response.content
if __name__ == '__main__':
app.run(port=PORT)
Plugin API responses should return raw data instead of natural language responses unless it’s necessary. ChatGPT will provide its own natural language response using the returned data.
{ "todos": [ "get groceries", "walk the dog" ] }
I have successfully installed wheel package. Moving on the next command for setup, that is,
pip install -r requirements
I found that there is no requirements file so that it could install the required libraries/packages.
Please guide me over it.
How can we enable CSRs to automate their workflows?
This task involves creating recordings involving one or more relevant applications, and implementing tests for corresponding completions:
We would like to implement the following functionality:
python puterbot/record.py "<description of the task to be recorded>"
For example:
python puterbot/record.py "a physician auditing billing codes (excel and powerchart)"
This should be saved to the Recording table, and passed into large language model as context.
Hello, I was trying to install MiniGPT 4, for that I had the run the command "conda env create -f environment.yml". This tried to install decord which gave me the error: "
Pip subprocess error:
ERROR: Could not find a version that satisfies the requirement decord==0.6.0 (from versions: none)
ERROR: No matching distribution found for decord==0.6.0
failed
CondaEnvException: Pip failed
(base) ahmethamamcioglu@Ahmet-MacBook-Air MiniGPT-4 % pip install --upgrade pip
Requirement already satisfied: pip in /Users/ahmethamamcioglu/anaconda3/lib/python3.10/site-packages (22.3.1)
Collecting pip
Downloading pip-23.1.2-py3-none-any.whl (2.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 14.2 MB/s eta 0:00:00
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 22.3.1
Uninstalling pip-22.3.1:
Successfully uninstalled pip-22.3.1
Successfully installed pip-23.1.2
(base) ahmethamamcioglu@Ahmet-MacBook-Air MiniGPT-4 % pip install decord==0.6.0
ERROR: Could not find a version that satisfies the requirement decord==0.6.0 (from versions: none)
ERROR: No matching distribution found for decord==0.6.0
(base) ahmethamamcioglu@Ahmet-MacBook-Air MiniGPT-4 % pip install decord
ERROR: Could not find a version that satisfies the requirement decord (from versions: none)
ERROR: No matching distribution found for decord" I also tried to install it separately but it did not work. I found this thread online: dmlc/decord#213 which did not have a solution for the same problem. I was wondering if anybody could help?
alembic upgrade head
FileNotFoundError: [Errno 2] No such file or directory: '/Users/taeyangkim/PAT/alembic/versions'
def take_screenshot(cls):
sct_img = take_screenshot()
screenshot = Screenshot(sct_img=sct_img)
return screenshot
In these lines (208-211) in Models.py, I'm confused on how this method works since there seems to be no base case ( as we call take_screenshot in the very first line ). I also checked the Base class in db.py to ensure if there was no method with the same name being inherited but this didn't seem to be the case. Just wondering how this works :D
How can we modify the DemoReplayStrategy that uses the available primitives (e.g. LLMs, OCR, SVG, etc) in a way that makes sense?
How can we take advantage of https://github.com/ai-hero/llm-tools ?
How can we enable Product Managers to automate their workflows?
This task involves creating recordings involving one or more relevant applications, and implementing tests for corresponding completions:
when I run gui_process_automation.py
command line (steps to reproduce)
/Users/puterbot/.venv/bin/python /Users/puterbot/puterbot/gui_process_automation.py
Exception
/Users/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py:121: DeprecationWarning: pkg_resources is deprecated as an API warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
/Users/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py:2870: DeprecationWarning: Deprecated call to pkg_resources.declare_namespace('mpl_toolkits').
Implementing implicit namespace packages (as specified in PEP 420) is preferred to pkg_resources.declare_namespace. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
/Users/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py:2870: DeprecationWarning: Deprecated call to pkg_resources.declare_namespace('google').
Implementing implicit namespace packages (as specified in PEP 420) is preferred to pkg_resources.declare_namespace. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
/Users/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py:2870: DeprecationWarning: Deprecated call to pkg_resources.declare_namespace('zope').
Implementing implicit namespace packages (as specified in PEP 420) is preferred to pkg_resources.declare_namespace. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
Downloading (…)olve/main/vocab.json: 100%|███| 798k/798k [00:00<00:00, 36.9MB/s]
Downloading (…)olve/main/merges.txt: 100%|███| 456k/456k [00:00<00:00, 11.8MB/s]
Downloading (…)in/added_tokens.json: 100%|█| 4.04k/4.04k [00:00<00:00, 1.83MB/s]
Downloading (…)cial_tokens_map.json: 100%|██████| 357/357 [00:00<00:00, 181kB/s]
Downloading (…)okenizer_config.json: 100%|██████| 619/619 [00:00<00:00, 249kB/s]
Downloading (…)lve/main/config.json: 100%|██████| 930/930 [00:00<00:00, 107kB/s]
Downloading pytorch_model.bin: 100%|███████| 24.2G/24.2G [22:01<00:00, 18.3MB/s]
. zsh: killed /Users/puterbot/.venv/bin/python
How can we enable people to automate their calendars?
This task involves creating recordings involving one or more relevant applications, and implementing tests for corresponding completions:
How can we enable individuals and/or businesses to automate their taxes:
This task involves creating recordings involving one or more relevant applications, and implementing tests for corresponding completions:
Hi,
I have a macOS and when I try to download all the requirements, it has an error and returns the following output. Specifically I input the following lines, with the last line generating the error:
Ignoring pywin32: markers 'sys_platform == "win32"' don't match your environment
Collecting git+https://github.com/abrichr/pynput.git (from -r requirements.txt (line 16))
Cloning https://github.com/abrichr/pynput.git to /private/var/folders/8s/xks38wt54rl3mjkykjhn6d_00000gn/T/pip-req-build-4aidr3_u
Running command git clone --filter=blob:none --quiet https://github.com/abrichr/pynput.git /private/var/folders/8s/xks38wt54rl3mjkykjhn6d_00000gn/T/pip-req-build-4aidr3_u
Resolved https://github.com/abrichr/pynput.git to commit 5d7256761b3cf6f01758c2d7d6a877d8facf79bb
Preparing metadata (setup.py) ... done
Collecting alembic==1.8.1 (from -r requirements.txt (line 1))
Using cached alembic-1.8.1-py3-none-any.whl (209 kB)
Collecting ascii_magic==2.3.0 (from -r requirements.txt (line 2))
Using cached ascii_magic-2.3.0-py3-none-any.whl (733 kB)
Collecting bokeh==2.4.3 (from -r requirements.txt (line 3))
Using cached bokeh-2.4.3-py3-none-any.whl (18.5 MB)
Collecting clipboard==0.0.4 (from -r requirements.txt (line 4))
Using cached clipboard-0.0.4-py3-none-any.whl
Collecting deepdiff==6.2.2 (from -r requirements.txt (line 5))
Using cached deepdiff-6.2.2-py3-none-any.whl (72 kB)
Collecting dictalchemy3==1.0.0 (from -r requirements.txt (line 6))
Using cached dictalchemy3-1.0.0-py3-none-any.whl
Collecting fire==0.4.0 (from -r requirements.txt (line 7))
Using cached fire-0.4.0-py2.py3-none-any.whl
Collecting ipdb==0.13.11 (from -r requirements.txt (line 8))
Using cached ipdb-0.13.11-py3-none-any.whl (12 kB)
Collecting loguru==0.6.0 (from -r requirements.txt (line 9))
Using cached loguru-0.6.0-py3-none-any.whl (58 kB)
Collecting matplotlib==3.6.2 (from -r requirements.txt (line 10))
Using cached matplotlib-3.6.2-cp310-cp310-macosx_11_0_arm64.whl (7.2 MB)
Collecting mss==6.1.0 (from -r requirements.txt (line 11))
Using cached mss-6.1.0-py3-none-any.whl (76 kB)
Collecting pandas==2.0.0 (from -r requirements.txt (line 12))
Using cached pandas-2.0.0-cp310-cp310-macosx_11_0_arm64.whl (10.8 MB)
Collecting pygetwindow==0.0.9 (from -r requirements.txt (line 13))
Using cached PyGetWindow-0.0.9-py3-none-any.whl
Collecting pyinstaller (from -r requirements.txt (line 14))
Using cached pyinstaller-5.10.1-py3-none-macosx_10_13_universal2.whl (925 kB)
Requirement already satisfied: pytest==7.1.3 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from -r requirements.txt (line 17)) (7.1.3)
Requirement already satisfied: rapidocr-onnxruntime==1.2.3 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from -r requirements.txt (line 18)) (1.2.3)
Requirement already satisfied: scikit-learn==1.2.2 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from -r requirements.txt (line 19)) (1.2.2)
Requirement already satisfied: scipy==1.9.3 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from -r requirements.txt (line 20)) (1.9.3)
Requirement already satisfied: sqlalchemy==1.4.43 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from -r requirements.txt (line 21)) (1.4.43)
Requirement already satisfied: torch==2.0.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from -r requirements.txt (line 22)) (2.0.0)
Requirement already satisfied: tqdm==4.64.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from -r requirements.txt (line 23)) (4.64.0)
Requirement already satisfied: transformers==4.28.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from -r requirements.txt (line 24)) (4.28.1)
Requirement already satisfied: Mako in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from alembic==1.8.1->-r requirements.txt (line 1)) (1.2.4)
Requirement already satisfied: colorama in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ascii_magic==2.3.0->-r requirements.txt (line 2)) (0.4.6)
Requirement already satisfied: Pillow in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ascii_magic==2.3.0->-r requirements.txt (line 2)) (9.5.0)
Requirement already satisfied: Jinja2>=2.9 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from bokeh==2.4.3->-r requirements.txt (line 3)) (3.1.2)
Requirement already satisfied: numpy>=1.11.3 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from bokeh==2.4.3->-r requirements.txt (line 3)) (1.24.3)
Requirement already satisfied: packaging>=16.8 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from bokeh==2.4.3->-r requirements.txt (line 3)) (23.1)
Requirement already satisfied: PyYAML>=3.10 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from bokeh==2.4.3->-r requirements.txt (line 3)) (6.0)
Requirement already satisfied: tornado>=5.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from bokeh==2.4.3->-r requirements.txt (line 3)) (6.3.1)
Requirement already satisfied: typing-extensions>=3.10.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from bokeh==2.4.3->-r requirements.txt (line 3)) (4.5.0)
Requirement already satisfied: pyperclip>=1.3 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from clipboard==0.0.4->-r requirements.txt (line 4)) (1.8.2)
Requirement already satisfied: ordered-set<4.2.0,>=4.0.2 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from deepdiff==6.2.2->-r requirements.txt (line 5)) (4.1.0)
Requirement already satisfied: six in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from fire==0.4.0->-r requirements.txt (line 7)) (1.16.0)
Requirement already satisfied: termcolor in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from fire==0.4.0->-r requirements.txt (line 7)) (2.3.0)
Requirement already satisfied: ipython>=7.31.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipdb==0.13.11->-r requirements.txt (line 8)) (8.13.0)
Requirement already satisfied: tomli in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipdb==0.13.11->-r requirements.txt (line 8)) (2.0.1)
Requirement already satisfied: decorator in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipdb==0.13.11->-r requirements.txt (line 8)) (5.1.1)
Requirement already satisfied: contourpy>=1.0.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from matplotlib==3.6.2->-r requirements.txt (line 10)) (1.0.7)
Requirement already satisfied: cycler>=0.10 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from matplotlib==3.6.2->-r requirements.txt (line 10)) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from matplotlib==3.6.2->-r requirements.txt (line 10)) (4.39.3)
Requirement already satisfied: kiwisolver>=1.0.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from matplotlib==3.6.2->-r requirements.txt (line 10)) (1.4.4)
Requirement already satisfied: pyparsing>=2.2.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from matplotlib==3.6.2->-r requirements.txt (line 10)) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from matplotlib==3.6.2->-r requirements.txt (line 10)) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pandas==2.0.0->-r requirements.txt (line 12)) (2023.3)
Requirement already satisfied: tzdata>=2022.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pandas==2.0.0->-r requirements.txt (line 12)) (2023.3)
Requirement already satisfied: pyrect in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pygetwindow==0.0.9->-r requirements.txt (line 13)) (0.2.0)
Requirement already satisfied: attrs>=19.2.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pytest==7.1.3->-r requirements.txt (line 17)) (23.1.0)
Requirement already satisfied: iniconfig in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pytest==7.1.3->-r requirements.txt (line 17)) (2.0.0)
Requirement already satisfied: pluggy<2.0,>=0.12 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pytest==7.1.3->-r requirements.txt (line 17)) (1.0.0)
Requirement already satisfied: py>=1.8.2 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pytest==7.1.3->-r requirements.txt (line 17)) (1.11.0)
Requirement already satisfied: pyclipper>=1.2.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from rapidocr-onnxruntime==1.2.3->-r requirements.txt (line 18)) (1.3.0.post4)
Requirement already satisfied: onnxruntime>=1.7.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from rapidocr-onnxruntime==1.2.3->-r requirements.txt (line 18)) (1.14.1)
Requirement already satisfied: opencv-python>=4.5.1.48 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from rapidocr-onnxruntime==1.2.3->-r requirements.txt (line 18)) (4.7.0.72)
Requirement already satisfied: Shapely>=1.7.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from rapidocr-onnxruntime==1.2.3->-r requirements.txt (line 18)) (2.0.1)
Requirement already satisfied: joblib>=1.1.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from scikit-learn==1.2.2->-r requirements.txt (line 19)) (1.2.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from scikit-learn==1.2.2->-r requirements.txt (line 19)) (3.1.0)
Requirement already satisfied: filelock in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from torch==2.0.0->-r requirements.txt (line 22)) (3.12.0)
Requirement already satisfied: sympy in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from torch==2.0.0->-r requirements.txt (line 22)) (1.11.1)
Requirement already satisfied: networkx in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from torch==2.0.0->-r requirements.txt (line 22)) (3.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from transformers==4.28.1->-r requirements.txt (line 24)) (0.14.1)
Requirement already satisfied: regex!=2019.12.17 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from transformers==4.28.1->-r requirements.txt (line 24)) (2023.3.23)
Requirement already satisfied: requests in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from transformers==4.28.1->-r requirements.txt (line 24)) (2.29.0)
Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from transformers==4.28.1->-r requirements.txt (line 24)) (0.13.3)
Requirement already satisfied: setuptools>=42.0.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pyinstaller->-r requirements.txt (line 14)) (58.1.0)
Requirement already satisfied: altgraph in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pyinstaller->-r requirements.txt (line 14)) (0.17.3)
Requirement already satisfied: pyinstaller-hooks-contrib>=2021.4 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pyinstaller->-r requirements.txt (line 14)) (2023.2)
Requirement already satisfied: macholib>=1.8 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pyinstaller->-r requirements.txt (line 14)) (1.16.2)
Requirement already satisfied: pyobjc-framework-ApplicationServices>=8.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pynput==1.7.6->-r requirements.txt (line 16)) (9.1.1)
Requirement already satisfied: pyobjc-framework-Quartz>=8.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pynput==1.7.6->-r requirements.txt (line 16)) (9.1.1)
Requirement already satisfied: fsspec in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.11.0->transformers==4.28.1->-r requirements.txt (line 24)) (2023.4.0)
Requirement already satisfied: backcall in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (0.2.0)
Requirement already satisfied: jedi>=0.16 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (0.18.2)
Requirement already satisfied: matplotlib-inline in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (0.1.6)
Requirement already satisfied: pickleshare in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (0.7.5)
Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (3.0.38)
Requirement already satisfied: pygments>=2.4.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (2.15.1)
Requirement already satisfied: stack-data in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (0.6.2)
Requirement already satisfied: traitlets>=5 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (5.9.0)
Requirement already satisfied: pexpect>4.3 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (4.8.0)
Requirement already satisfied: appnope in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (0.1.3)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from Jinja2>=2.9->bokeh==2.4.3->-r requirements.txt (line 3)) (2.1.2)
Requirement already satisfied: coloredlogs in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from onnxruntime>=1.7.0->rapidocr-onnxruntime==1.2.3->-r requirements.txt (line 18)) (15.0.1)
Requirement already satisfied: flatbuffers in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from onnxruntime>=1.7.0->rapidocr-onnxruntime==1.2.3->-r requirements.txt (line 18)) (23.3.3)
Requirement already satisfied: protobuf in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from onnxruntime>=1.7.0->rapidocr-onnxruntime==1.2.3->-r requirements.txt (line 18)) (4.22.3)
Requirement already satisfied: pyobjc-core>=9.1.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pyobjc-framework-ApplicationServices>=8.0->pynput==1.7.6->-r requirements.txt (line 16)) (9.1.1)
Requirement already satisfied: pyobjc-framework-Cocoa>=9.1.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pyobjc-framework-ApplicationServices>=8.0->pynput==1.7.6->-r requirements.txt (line 16)) (9.1.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from requests->transformers==4.28.1->-r requirements.txt (line 24)) (3.1.0)
Requirement already satisfied: idna<4,>=2.5 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from requests->transformers==4.28.1->-r requirements.txt (line 24)) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from requests->transformers==4.28.1->-r requirements.txt (line 24)) (1.26.15)
Requirement already satisfied: certifi>=2017.4.17 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from requests->transformers==4.28.1->-r requirements.txt (line 24)) (2022.12.7)
Requirement already satisfied: mpmath>=0.19 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from sympy->torch==2.0.0->-r requirements.txt (line 22)) (1.3.0)
Requirement already satisfied: parso<0.9.0,>=0.8.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from jedi>=0.16->ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (0.8.3)
Requirement already satisfied: ptyprocess>=0.5 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from pexpect>4.3->ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (0.7.0)
Requirement already satisfied: wcwidth in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (0.2.6)
Requirement already satisfied: humanfriendly>=9.1 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from coloredlogs->onnxruntime>=1.7.0->rapidocr-onnxruntime==1.2.3->-r requirements.txt (line 18)) (10.0)
Requirement already satisfied: executing>=1.2.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from stack-data->ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (1.2.0)
Requirement already satisfied: asttokens>=2.1.0 in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from stack-data->ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (2.2.1)
Requirement already satisfied: pure-eval in /Users/d/rongdian/.venv/lib/python3.10/site-packages (from stack-data->ipython>=7.31.1->ipdb==0.13.11->-r requirements.txt (line 8)) (0.2.2)
Building wheels for collected packages: pynput
Building wheel for pynput (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [28 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/private/var/folders/8s/xks38wt54rl3mjkykjhn6d_00000gn/T/pip-req-build-4aidr3_u/setup.py", line 87, in <module>
setuptools.setup(
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/__init__.py", line 152, in setup
_install_setup_requires(attrs)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 806, in fetch_build_eggs
resolved_dists = pkg_resources.working_set.resolve(
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 766, in resolve
dist = best[req.key] = env.best_match(
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1051, in best_match
return self.obtain(req, installer)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1063, in obtain
return installer(requirement)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 877, in fetch_build_egg
return fetch_build_egg(self, req)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/installer.py", line 80, in fetch_build_egg
wheel.install_as_egg(dist_location)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 95, in install_as_egg
self._install_as_egg(destination_eggdir, zf)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 103, in _install_as_egg
self._convert_metadata(zf, destination_eggdir, dist_info, egg_info)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 124, in _convert_metadata
os.mkdir(destination_eggdir)
FileExistsError: [Errno 17] File exists: '/private/var/folders/8s/xks38wt54rl3mjkykjhn6d_00000gn/T/pip-req-build-4aidr3_u/.eggs/wrapt-1.15.0-py3.10-macosx-10.9-universal2.egg'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pynput
Running setup.py clean for pynput
error: subprocess-exited-with-error
× python setup.py clean did not run successfully.
│ exit code: 1
╰─> [28 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/private/var/folders/8s/xks38wt54rl3mjkykjhn6d_00000gn/T/pip-req-build-4aidr3_u/setup.py", line 87, in <module>
setuptools.setup(
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/__init__.py", line 152, in setup
_install_setup_requires(attrs)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 806, in fetch_build_eggs
resolved_dists = pkg_resources.working_set.resolve(
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 766, in resolve
dist = best[req.key] = env.best_match(
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1051, in best_match
return self.obtain(req, installer)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1063, in obtain
return installer(requirement)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 877, in fetch_build_egg
return fetch_build_egg(self, req)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/installer.py", line 80, in fetch_build_egg
wheel.install_as_egg(dist_location)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 95, in install_as_egg
self._install_as_egg(destination_eggdir, zf)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 103, in _install_as_egg
self._convert_metadata(zf, destination_eggdir, dist_info, egg_info)
File "/Users/d/rongdian/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 124, in _convert_metadata
os.mkdir(destination_eggdir)
FileExistsError: [Errno 17] File exists: '/private/var/folders/8s/xks38wt54rl3mjkykjhn6d_00000gn/T/pip-req-build-4aidr3_u/.eggs/wrapt-1.15.0-py3.10-macosx-10.9-universal2.egg'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed cleaning build dir for pynput
Failed to build pynput
ERROR: Could not build wheels for pynput, which is required to install pyproject.toml-based projects
I want to import models to have the access to the InputEvent object. Please let me know if this is how we have import or InputEvent.
Another way i can think of is creating namedTuple as seen record.py as well:
from typing import List
from collections import namedtuple
InputEvent = namedtuple('InputEvent', ['type', 'properties'])
Screenshot = namedtuple('Screenshot', ['png_data'])
WindowEvent = namedtuple('WindowEvent', ['type', 'properties'])Please guide me. Thank you.
Building wheels for collected packages: pynput
Building wheel for pynput (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [30 lines of output]
/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
warnings.warn(
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/private/var/folders/sz/cmfct53971d47n8gp7wk77280000gn/T/pip-req-build-hwg9sldu/setup.py", line 87, in
setuptools.setup(
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/init.py", line 86, in setup
_install_setup_requires(attrs)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/init.py", line 80, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 875, in fetch_build_eggs
resolved_dists = pkg_resources.working_set.resolve(
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 789, in resolve
dist = best[req.key] = env.best_match(
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 1075, in best_match
return self.obtain(req, installer)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 1087, in obtain
return installer(requirement)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 945, in fetch_build_egg
return fetch_build_egg(self, req)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/installer.py", line 87, in fetch_build_egg
wheel.install_as_egg(dist_location)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 110, in install_as_egg
self._install_as_egg(destination_eggdir, zf)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 118, in _install_as_egg
self._convert_metadata(zf, destination_eggdir, dist_info, egg_info)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 162, in _convert_metadata
os.rename(dist_info, egg_info)
OSError: [Errno 66] Directory not empty: '/private/var/folders/sz/cmfct53971d47n8gp7wk77280000gn/T/pip-req-build-hwg9sldu/.eggs/charset_normalizer-3.1.0-py3.10-macosx-10.9-universal2.egg/charset_normalizer-3.1.0.dist-info' -> '/private/var/folders/sz/cmfct53971d47n8gp7wk77280000gn/T/pip-req-build-hwg9sldu/.eggs/charset_normalizer-3.1.0-py3.10-macosx-10.9-universal2.egg/EGG-INFO'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pynput
Running setup.py clean for pynput
error: subprocess-exited-with-error
× python setup.py clean did not run successfully.
│ exit code: 1
╰─> [30 lines of output]
/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
warnings.warn(
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/private/var/folders/sz/cmfct53971d47n8gp7wk77280000gn/T/pip-req-build-hwg9sldu/setup.py", line 87, in
setuptools.setup(
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/init.py", line 86, in setup
_install_setup_requires(attrs)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/init.py", line 80, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 875, in fetch_build_eggs
resolved_dists = pkg_resources.working_set.resolve(
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 789, in resolve
dist = best[req.key] = env.best_match(
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 1075, in best_match
return self.obtain(req, installer)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 1087, in obtain
return installer(requirement)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 945, in fetch_build_egg
return fetch_build_egg(self, req)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/installer.py", line 87, in fetch_build_egg
wheel.install_as_egg(dist_location)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 110, in install_as_egg
self._install_as_egg(destination_eggdir, zf)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 118, in _install_as_egg
self._convert_metadata(zf, destination_eggdir, dist_info, egg_info)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 162, in _convert_metadata
os.rename(dist_info, egg_info)
OSError: [Errno 66] Directory not empty: '/private/var/folders/sz/cmfct53971d47n8gp7wk77280000gn/T/pip-req-build-hwg9sldu/.eggs/charset_normalizer-3.1.0-py3.10-macosx-10.9-universal2.egg/charset_normalizer-3.1.0.dist-info' -> '/private/var/folders/sz/cmfct53971d47n8gp7wk77280000gn/T/pip-req-build-hwg9sldu/.eggs/charset_normalizer-3.1.0-py3.10-macosx-10.9-universal2.egg/EGG-INFO'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed cleaning build dir for pynput
Failed to build pynput
Installing collected packages: wcwidth, pytz, pyrect, pyperclip, pure-eval, ptyprocess, pickleshare, executing, backcall, appnope, altgraph, tzdata, typing-extensions, traitlets, tqdm, tornado, tomli, termcolor, six, PyYAML, pyparsing, pyobjc-core, pyinstaller-hooks-contrib, pygments, pygetwindow, py, prompt-toolkit, pluggy, pillow, pexpect, parso, packaging, ordered-set, numpy, mss, MarkupSafe, macholib, loguru, kiwisolver, iniconfig, greenlet, fonttools, decorator, cycler, clipboard, attrs, sqlalchemy, scipy, python-dateutil, pytest, pyobjc-framework-Cocoa, pyinstaller, matplotlib-inline, Mako, Jinja2, jedi, fire, deepdiff, contourpy, asttokens, stack-data, pyobjc-framework-Quartz, pandas, matplotlib, dictalchemy3, bokeh, alembic, pyobjc-framework-ApplicationServices, ipython, pynput, ipdb
Running setup.py install for pynput ... error
error: subprocess-exited-with-error
× Running setup.py install for pynput did not run successfully.
│ exit code: 1
╰─> [30 lines of output]
/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
warnings.warn(
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/private/var/folders/sz/cmfct53971d47n8gp7wk77280000gn/T/pip-req-build-hwg9sldu/setup.py", line 87, in
setuptools.setup(
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/init.py", line 86, in setup
_install_setup_requires(attrs)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/init.py", line 80, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 875, in fetch_build_eggs
resolved_dists = pkg_resources.working_set.resolve(
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 789, in resolve
dist = best[req.key] = env.best_match(
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 1075, in best_match
return self.obtain(req, installer)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/pkg_resources/init.py", line 1087, in obtain
return installer(requirement)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 945, in fetch_build_egg
return fetch_build_egg(self, req)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/installer.py", line 87, in fetch_build_egg
wheel.install_as_egg(dist_location)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 110, in install_as_egg
self._install_as_egg(destination_eggdir, zf)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 118, in _install_as_egg
self._convert_metadata(zf, destination_eggdir, dist_info, egg_info)
File "/Users/taeyangkim/PAT/puterbot/.venv/lib/python3.10/site-packages/setuptools/wheel.py", line 162, in _convert_metadata
os.rename(dist_info, egg_info)
OSError: [Errno 66] Directory not empty: '/private/var/folders/sz/cmfct53971d47n8gp7wk77280000gn/T/pip-req-build-hwg9sldu/.eggs/charset_normalizer-3.1.0-py3.10-macosx-10.9-universal2.egg/charset_normalizer-3.1.0.dist-info' -> '/private/var/folders/sz/cmfct53971d47n8gp7wk77280000gn/T/pip-req-build-hwg9sldu/.eggs/charset_normalizer-3.1.0-py3.10-macosx-10.9-universal2.egg/EGG-INFO'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure
× Encountered error while trying to install package.
╰─> pynput
note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
How can we implement crowdfunding functionality?
This is the error I get after running python puterbot/visualize.py:
inside this function of the crud.py file
I gave my pc permission to record the screen, as I initially thought that was the issue, but the error still persists :( Could use some help
(Originally posted at https://www.linkedin.com/posts/richard-abrich_httpslnkdinepbaczqj-activity-7059302632053469184-uXNb)
Announcing Machine Learning, Data Science, Artificial Intelligence: Conceptualization to Commercialization.
In this first series of articles, every word is written by ChatGPT (via repeated prompting). It's for everyone who wants to learn about Generative Pretrained Transformers, and how you can start using them to automate your processes today. Especially if you have no prior programming or technical background.
We will post tutorials for people with no prior programming experience to start automating their processes with our open source software library, https://github.com/MLDSAI/puterbot.
In addition, we are sponsoring prizes for the Tribe AI Large Language Model Hackathon. The deadline to register is tomorrow, Wednesday May 3rd. It's ten days long, part time. Contribute as much (or as little as an hour or two) as you are able. Sign up here: https://lu.ma/6qm0or78 (@Mustaballer @jesicasusanto @ahmetutku @angelala3252 @dianzrong @micol-altomare)
We are looking for a) machine learning and software practitioners, and b) people who have limited/no programming experience. You will be our early users, and by joining us in this early phase, you can shape the application of Large Language Models (LLMs) and other Generative Pretrained Transformers (GPTs) to 10x'ing productivity in your industry.
Stay tuned! Subscribe to MLDSAI.substack.com and PuterBot.com for more.
#machinelearning #datascience #artificialintelligence
#productivity #opensource #processautomation #ai #hackathon2023 🚀
--
See https://github.com/MLDSAI/PuterBot/issues?q=is%3Aissue+is%3Aopen+label%3Atribe-hackathon for ideas on where to start.
Please feel free to leave a comment here to introduce yourself and share what's top of mind for you!
We want to use the most powerful model on the planet via the Openai API
We would like to convert Screenshots and InputEvents to a text representation (e.g. via ASCII and OCR) and then feed this text representation into a Large Language Model (e.g. https://gpt4all.io), given a text description of the goal to be accomplished.
Hi,
I wanted to let you know that I encountered an error while running the record.py script in the puterbot project. When I tried running the script with the command "python record.py 'testing out puterbot'", I got the following error message:
It looks like the issue is with the get_screenshot function from the puterbot.utils module. The error message says that it couldn't find the get_screenshot function in the utils.py file. Is get_screenshot part of the function we should implement?
Any help would be greatly appreciated. Thanks!
How can we take advantage of https://python.langchain.com/en/latest/getting_started/getting_started.html ?
How can we make it extremely simple for people to create and share use cases?
Capture user narration via speech to text e.g. https://github.com/openai/whisper
Hello,
I was trying to set up everything in MacOS. I removed "pywin32==306" from the requirements.txt file instead of the readme file since it is not compatible with my Mac. However, I have been getting errors (as seen in the images attached).
Certain use cases (e.g. SalesForce, LinkedIn) would greatly benefit from reading and writing website markup and the DOM (i.e. by simplifying tokenization of screenshots).
Implement a Chrome Plugin that sends and receives messages from PuterBot via native messaging: https://developer.chrome.com/docs/extensions/mv3/nativeMessaging/ . On every change to the Document Object Model (DOM) of the currently opened page, the extension should send a message to PuterBot.
Implement an adapter in PuterBot to receive and store the messages with https://github.com/Rayquaza01/nativemessaging. To start with, this can look something like this:
browser.py:
import nativemessaging
def main():
# based on https://github.com/Rayquaza01/nativemessaging#sample
reply_num = 0
while True:
message = nativemessaging.get_message()
print(message)
nativemessaging.send_message(nativemessaging.encode_message(str(reply_num))
reply_num += 1
if __name__ == "__main__":
main()
In the extension:
// from https://github.com/Rayquaza01/nativemessaging#sample
function onReceived(response) {
console.log(response);
}
// runtime.connectNative
var port = browser.runtime.connectNative("application_name");
port.onMessage.addListener(onReceived);
port.postMessage("hello");
// runtime.sendNativeMessage
browser.runtime.sendNativeMessage("puterbot", document).then(onReceived);
// based on https://stackoverflow.com/questions/8882502/how-to-track-dom-change-in-chrome-extension
let observer = new MutationObserver(mutations => {
for(let mutation of mutations) {
browser.runtime.sendNativeMessage("puterbot", mutation).then(onReceived);
}
});
observer.observe(document, { childList: true, subtree: true });
Once this is working, we will want to add the necessary functionality to record.py.
How can we take advantage of https://github.com/microsoft/TaskMatrix ?
This task involves undestanding whether long term memory is necessary, and if so, building a minimum viable implementation, e.g:
pg_vector:
https://github.com/pgvector/pgvector
https://www.youtube.com/watch?v=aywZrzNaKjs&t=539s
https://news.ycombinator.com/item?id=35311113
Supabase? https://supabase.com/
Pinecone:
https://python.langchain.com/en/latest/ecosystem/pinecone.html
Weaviate
We would like to make it easy to scrub private information (e.g. PII/PHI):
python -m openadapt.scrub
How can we enable business managers to automate their workflows?
This task involves creating recordings involving one or more relevant applications, and implementing tests for corresponding completions:
We would like to convert Screenshots and InputEvents to a text representation (e.g. via Segment Anything and OCR) and then feed this text representation into a Large Language Model (e.g. https://gpt4all.io/) given a text description of the goal to be accomplished.
Occasionally, attempting to replay or visualize a recording will result in the following error:
2023-04-22 19:23:42.140 | INFO | puterbot.events:merge_consecutive_input_events:570 - name='mouse_move' num_events_removed=370
2023-04-22 19:23:42.140 | ERROR | puterbot.events:process_events:621 - (<function merge_consecutive_mouse_move_events at 0x13632f0a0>, InputEvent(id=2778, name='type', timestamp=1683820640.0885906, recording_timestamp=1682204625.970328, screenshot_timestamp=1683820641.2495313, window_event_timestamp=1683820634.120639, text='h-i-s', canonical_text='h-i-s'), InputEvent(id=2779, name='type', timestamp=1683820639.0881405, recording_timestamp=1682204625.970328, screenshot_timestamp=1683820641.4406426, window_event_timestamp=1683820634.120639, text='<space>-i-s', canonical_text='<49>-i-s'))
Traceback (most recent call last):
File "/Users/abrichr/MLDSAI/src/puterbot/puterbot/visualize.py", line 226, in <module>
main()
└ <function main at 0x13b1f7520>
File "/Users/abrichr/MLDSAI/src/puterbot/puterbot/visualize.py", line 135, in main
input_events = get_events(recording, process=PROCESS_EVENTS, meta=meta)
│ │ │ └ {}
│ │ └ True
│ └ Recording(id=14, timestamp=1682204625.970328, monitor_width=1512, monitor_height=982, double_click_interval_seconds=0.5, doub...
└ <function get_events at 0x13632ed40>
File "/Users/abrichr/MLDSAI/src/puterbot/puterbot/events.py", line 53, in get_events
input_events, window_events, screenshots = process_events(
│ │ └ <function process_events at 0x13b1f72e0>
│ └ [WindowEvent(id=24, recording_timestamp=1682204625.970328, timestamp=1683820634.120639, title='Terminal puterbot — Python ◂ P...
└ [InputEvent(id=2366, name='move', timestamp=1683820639.0447876, recording_timestamp=1682204625.970328, screenshot_timestamp=1...
> File "/Users/abrichr/MLDSAI/src/puterbot/puterbot/events.py", line 617, in process_events
assert prev_event.timestamp <= event.timestamp, (
│ │ │ └ <sqlalchemy.orm.attributes.InstrumentedAttribute object at 0x13633b830>
│ │ └ InputEvent(id=2779, name='type', timestamp=1683820639.0881405, recording_timestamp=1682204625.970328, screenshot_timestamp=16...
│ └ <sqlalchemy.orm.attributes.InstrumentedAttribute object at 0x13633b830>
└ InputEvent(id=2778, name='type', timestamp=1683820640.0885906, recording_timestamp=1682204625.970328, screenshot_timestamp=16...
AssertionError: (<function merge_consecutive_mouse_move_events at 0x13632f0a0>, InputEvent(id=2778, name='type', timestamp=1683820640.0885906, recording_timestamp=1682204625.970328, screenshot_timestamp=1683820641.2495313, window_event_timestamp=1683820634.120639, text='h-i-s', canonical_text='h-i-s'), InputEvent(id=2779, name='type', timestamp=1683820639.0881405, recording_timestamp=1682204625.970328, screenshot_timestamp=1683820641.4406426, window_event_timestamp=1683820634.120639, text='<space>-i-s', canonical_text='<49>-i-s'))
> /Users/abrichr/MLDSAI/src/puterbot/puterbot/events.py(615)process_events()
614 # TODO: keep events in which window_event_timestamp is updated
--> 615 for prev_event, event in zip(input_events, input_events[1:]):
616 try:
ipdb>
Steps to reproduce:
python puterbot/record.py "testing out puterbot"python puterbot/visualize.py or python puterbot/replay.py NaiveReplayStratevyWe would like to convert Screenshots and InputEvents to a text representation (e.g. via SVG and OCR) and then feed this text representation into a Large Language Model (e.g. https://gpt4all.io) given a text description of the goal to be accomplished.
After a while, python puterbot/record.py will cause the system to run out of memory.
We would like to provide strategies with known targets in order to measure their performance and help users guide their agents (see #42).
Therefore we would like to implement "signals". Example:
from openadapt import signals
# file
signals.add_signal(file_path="<path>/<to>/<file>.[json|csv|...])
# database
signals.add_signal(db_url="pgsql://<username>:<password>@<host>:<port>/[<name>]")
# url
signals.add_signal(http_url="https://...") # e.g. platform.openai.com/...
# python function
signals.add_signal("openadapt.signal.openai.<...>")
signals.add_signal("openadapt.signal.contrib.<...>")
I encountered an error message while attempting to complete the last setup step for PuterBot, which involves running the command "alembic upgrade head". The error message indicated that Alembic could not locate its configuration file named "alembic.ini"
Steps to reproduce : same as puterbot setup.
However, I found the "alembic.ini" file in the PAT directory, so I tried moving to that directory and running the "alembic upgrade head" command again. This time, the command ran without error.
I would like to confirm whether it is acceptable to perform the setup for PuterBot by moving to the PAT directory first before running the "alembic upgrade head" command. Thank you!
We would like to show ScreenShots and InputEvents to MiniGPT-4 (https://minigpt-4.github.io/), and have it generate the next InputEvent, given a text description of the goal to be accomplished. OCR will likely also be necessary here.
How can we enable marketers to automate their workflows?
This task involves creating recordings involving one or more relevant applications, and implementing tests for corresponding completions:
Marketo
Salesforce
Thrift
Hubspot
Active campaigns
Facebook Messenger
How can we extend DemoReplayStrategy to use https://github.com/ShreyaR/guardrails ?
When run with pytest, most tests are failing on Windows 10 due to the timestamps not matching.
Windows 10
python 3.10.11
Example mismatch in test assert:
E 'timestamp': 5.775},
E - {'children': [{'mouse_pressed': True, 'timestamp': 11.88},
E + {'children': [{'mouse_pressed': True, 'timestamp': 11.880000000000003},
E ? +++++++++++++
E - {'mouse_pressed': False, 'timestamp': 12.155000000000001},
E ? ^
E + {'mouse_pressed': False, 'timestamp': 12.155000000000003},
E ? ^
E - {'mouse_pressed': True, 'timestamp': 12.21},
E + {'mouse_pressed': True, 'timestamp': 12.210000000000003},
E ? +++++++++++++
E - {'mouse_pressed': False, 'timestamp': 12.485000000000001}],
E ? ^
E + {'mouse_pressed': False, 'timestamp': 12.485000000000003}],
E ? ^
E 'mouse_button_name': 'left',
E 'name': 'doubleclick',
E 'timestamp': 11.275},
E {'mouse_button_name': 'right',
E 'mouse_pressed': True,
E 'name': 'click',
E 'timestamp': 16.775},
E {'mouse_button_name': 'right',
E 'mouse_pressed': False,
E 'name': 'click',
E 'timestamp': 17.049999999999997},
E - {'children': [{'mouse_pressed': True, 'timestamp': 23.759999999999998},
E ? ^^^^^^^^^^^^^^
E + {'children': [{'mouse_pressed': True, 'timestamp': 23.76},
E ? ^
E - {'mouse_pressed': False, 'timestamp': 24.034999999999997}],
E ? ^^^^^^^^^^^^^
E + {'mouse_pressed': False, 'timestamp': 24.035}],
E ?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.