Comments (8)
@micol-altomare you might enjoy this one 😄
from openadapt.
Solution 1: Record videos with Loom
Benefits:
- users don't need to create an account or download anything else to create recordings
- recordings can be easily shared using a url
Implementation:
- starter code is given on Loom but is written in React, Typescript or JavaScript so would also need some Python backend
Limitations:
- under the free account, guest accounts are restricted to a maximum number of 5 videos, and maximum duration of 5 minutes
from openadapt.
Solution 2: using a Python library
Benefits:
- doesn't require another language
Implementation:
- most commonly, PyAutoGUI is used to take screenshots of the user's screen and OpenCV is used to take video from the user's camera
Limitations:
- needs the user to run the program in the command-line
- since PyAutoGUI is taking screenshots and it has a very slow frame rate, the video quality is very low
from openadapt.
Solution 3: using Ziggeo API
Benefits:
- has a Python SDK
- has built-in audio transcription capabilities
- can easily upload to youtube
- can put our own branding on it
Implementation:
- use their Python SDK
Limitations:
- may not be able to control the quality of the video (if the default/WebRTC recording method is used)
from openadapt.
Can also use whisper in combination with the above solutions to communicate use-cases without having to demonstrate it
Benefit:
- more accessible (for those who can't or find typing/clicking painful)
- also more human-friendly
Limitation:
- can be hard to line up audio and actions
from openadapt.
Thanks for taking the initiative on this @dianzrong !
Solution 1: Record videos with Loom
I think this is preferable since Loom is the most popular and easy to use screen casting utility AFAIK. Here are some sources that might be helpful:
https://github.com/Ecno92/loom-dl
ytdl-org/youtube-dl#27957
Solution 2: using a Python library
We considered using PyAutoGUI to implement OpenAdapt, but ended up implementing it from scratch for various reasons. I think we would not want to go this route unless there is a compelling reason.
Solution 3: using Ziggeo API
This could be interesting, I haven't come across Ziggeo before. A glance at their pricing suggests they don't have a free tier however, which makes this less attractive.
Other options:
https://github.com/jonnekaunisto/simple-youtube-api
Uploading to YouTube might be the easiest to start. Eventually I think we also want to support Loom, but their docs suggest this can only be done from a web application. If that's the case then https://nicegui.io/ may also be relevant. It should be possible to interact with their HTTP API directly, e.g. using Python's requests
library.
@dianzrong how comfortable are you with web development? Which approach sounds most interesting?
from openadapt.
@abrichr I do not have experience with web development but would love to spend time on it. Also I feel like I might as well get started with Loom since we want it later. For now I think it would be fastest to get something working using niceGUI so I will start with that!
from openadapt.
from openadapt.
Related Issues (20)
- Extract Process API from Process Graph HOT 1
- Implement Replay Action Confirmation
- Implement `openadapt.api.to_powerautomate(recording)`
- Deploy PromptFlow to AWS HOT 4
- [Bug]: Mouse flickers during capture on Windows devices
- Implement VISPROG
- Implement `openadapt.adapters.groq`
- Avoid unnecessary segmentation + description in `VisualReplayStrategy` HOT 1
- Implement Instructor for structured LLM outputs
- Anthropic image descriptions out of order
- Implement Reka Core adapter
- Implement NanoLLM completion adapter
- Implement NanoLLaVA completion adapter
- Implement Autodistill
- Support winget HOT 1
- [Bug]: manual installation on Windows fails on `poetry shell`
- please make docker type installation for ease. HOT 2
- please add python3.12 support HOT 2
- please add install instructions for linux on the `https://openadapt.ai/#start` page HOT 2
- [Bug]: from matplotlib._path import ( ImportError: DLL load failed while importing _path: The specified module could not be found.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openadapt.