the-full-stack / ask-fsdl Goto Github PK
View Code? Open in Web Editor NEWDocument Q&A over The Full Stack's Corpus
Home Page: https://fsdl.me/join-discord-askfsdl
License: MIT License
Document Q&A over The Full Stack's Corpus
Home Page: https://fsdl.me/join-discord-askfsdl
License: MIT License
ideally, we'd make it possible to swap them easily
switch to command instead of prefixes, other assorted changes
QoL improvement for folks who want to use/adapt this
Switched to ask-fsdl-llm
while we were maintaining two separate instances, should switch it back
Chunks should include contextual info at the top, like the document and section title, when they're presented to the LM.
This can be stored as metadata and inserted at prompt construction time.
probably due to not handling Discord's async behavior correctly, we only error out on timeout, even once the server has 500'd
we should at the very least return the 500s once we have them.
error handling in general is minimal. retry logic with backoff would be even better, but let's hold off. maybe the new Gradio SDK would give us those features easily?
the source field is a useful one to search over, usually with regex
prefix-matching regexes in Mongo can use lexical text indices
vector search is a type of index, so we want to have more (simpler) indices to point to when teaching this material
this is weirdly hard -- JSON doesn't show up nicely in VS Code Jupyter; JSON in the terminal has been a PITA with shell-specific nonsense
private Colab here with turbojank implementation
https://openai.com/blog/function-calling-and-other-api-updates
tl;dr: 16k context windows for turbo, better system prompting (move from user prompt to system?), tool use that might actually work
see this makefile for an example
could do AWS by hand or could try out Pulumi -- relatively simple.
For starters, just need to share a handful of secrets and pull down the repo on a single instance, then run some make
commands.
the developer experience is just OK. let's make it awesome.
note: some of these are more suggestion than requirement
pyenv virtualenv
seems to be on the out, see VS Code docsmodal serve
in development so that the backend endpoint isn't polluted -- also gets faster updates
debug
command to use modal shell
/
mount and the /gradio
mount/health
a "hidden command" so we can use it in prodinteractions
library for Discord to see if it's cleaner, especially for streaming responsesexample from pycord docs: https://docs.pycord.dev/en/stable/faq.html#what-does-blocking-mean
waiting on the backend is the majority of the execution time, so this is pretty important
the same way we've abstracted vecstore
and docstore
The LLM paper corpus code lives elsewhere, should bring it in here.
I could not find a way to reopen so I opened new.
"reopen" #57 (comment)
git --no-pager log -n 1
commit d7a12fb (HEAD -> main, upstream/main, origin/main)
Author: Charles Frye [email protected]
Date: Sun Jun 18 02:08:14 2023 +0000
removes git branch selection now that pulumi is in main
make document-store
python -m pip install -qqq -r requirements.txt
Verifying token against https://api.modal.com
Token verified successfully
Token written to /home/ido/.modal.toml
bash tasks/send_secrets_to_modal.sh
Created a new secret 'mongodb-fsdl' with the keys 'MONGODB_USER', 'MONGODB_URI', 'MONGODB_PASSWORD'
Use it in to your Modal app using:
@stub.function(secret=modal.Secret.from_name("mongodb-fsdl"))
def some_function():
os.getenv("MONGODB_USER")
os.getenv("MONGODB_URI")
os.getenv("MONGODB_PASSWORD")
Created a new secret 'openai-api-key-fsdl' with the key 'OPENAI_API_KEY'
Use it in to your Modal app using:
@stub.function(secret=modal.Secret.from_name("openai-api-key-fsdl"))
def some_function():
os.getenv("OPENAI_API_KEY")
Created a new secret 'gantry-api-key-fsdl' with the key 'GANTRY_API_KEY'
Use it in to your Modal app using:
@stub.function(secret=modal.Secret.from_name("gantry-api-key-fsdl"))
def some_function():
os.getenv("GANTRY_API_KEY")
tasks/run_etl.sh --drop --db fsdl-dev --collection ask-fsdl
โ Initialized. View app at https://modal.com/apps/ap-mcra9lE85EmUQPHM7Nuj2V
โ Created objects.
โโโ ๐จ Created web => https://ido777--ask-fsdl-hook-dev.modal.run
โโโ ๐จ Created mount /home/ido/gGPT/ask-fsdl/vecstore.py
โโโ ๐จ Created mount /home/ido/gGPT/ask-fsdl/docstore.py
โโโ ๐จ Created mount /home/ido/gGPT/ask-fsdl/utils.py
โโโ ๐จ Created mount /home/ido/gGPT/ask-fsdl/prompts.py
โโโ ๐จ Created mount /home/ido/gGPT/ask-fsdl/app.py
โโโ ๐จ Created mount /home/ido/gGPT/ask-fsdl/utils.py
โโโ ๐จ Created mount /home/ido/gGPT/ask-fsdl/vecstore.py
โโโ ๐จ Created qanda_langchain.
โโโ ๐จ Created create_vector_index.
โโโ ๐จ Created drop_docs.
โโโ ๐จ Created cli.
โโโ ๐จ Created fastapi_app => https://ido777--ask-fsdl-dev.modal.run
Traceback (most recent call last):
File "/pkg/modal/_container_entrypoint.py", line 330, in handle_input_exception
yield
File "/pkg/modal/_container_entrypoint.py", line 403, in call_function_sync
res = fun(*args, **kwargs)
File "/root/app.py", line 177, in drop_docs
docstore.drop(collection, db)
File "/pkg/docstore.py", line 17, in drop
collection = get_collection(collection, db, client)
File "/pkg/docstore.py", line 44, in get_collection
db = get_database(db, client)
File "/pkg/docstore.py", line 59, in get_database
client = client or connect()
File "/pkg/docstore.py", line 86, in connect
client = pymongo.MongoClient(connection_string, connect=True, appname="ask-fsdl")
File "/usr/local/lib/python3.10/site-packages/pymongo/mongo_client.py", line 639, in init
res = uri_parser.parse_uri(
File "/usr/local/lib/python3.10/site-packages/pymongo/uri_parser.py", line 461, in parse_uri
raise InvalidURI('Bad database name "%s"' % dbase)
pymongo.errors.InvalidURI: Bad database name "/user:password@fsdl"
The real user and password value were replaced
I tried to install it with no luck.
I progressed a bit and found some errors I the guide.
I will open a PR for the errors I already overcome, but now I am stuck on
make document-store
ExecutionError: Could not deserialize remote exception due to local error:
No module named 'pymongo'
This can happen if your local environment does not have the remote exception definitions.
Here is the remote traceback:
Traceback (most recent call last):
File "/pkg/modal/_container_entrypoint.py", line 330, in handle_input_exception
yield
File "/pkg/modal/_container_entrypoint.py", line 403, in call_function_sync
res = fun(*args, **kwargs)
File "/root/app.py", line 176, in drop_docs
docstore.drop(collection, db)
File "/pkg/docstore.py", line 17, in drop
collection = get_collection(collection, db, client)
File "/pkg/docstore.py", line 44, in get_collection
db = get_database(db, client)
File "/pkg/docstore.py", line 59, in get_database
client = client or connect()
File "/pkg/docstore.py", line 81, in connect
client = pymongo.MongoClient(connection_string, connect=True, appname="ask-fsdl")
File "/usr/local/lib/python3.10/site-packages/pymongo/mongo_client.py", line 639, in init
res = uri_parser.parse_uri(
File "/usr/local/lib/python3.10/site-packages/pymongo/uri_parser.py", line 461, in parse_uri
raise InvalidURI('Bad database name "%s"' % dbase)
pymongo.errors.InvalidURI: Bad database name "/<my_user>:@fsdl"
after completing #51
right now, just set heuristically
MVP: update the list of YouTube videos and rerun notebook ETL
Stretch: move the ETL into Modal and run it from the notebook
automatically generating it will allow it to keep up with changes in the structure and to survive if we move to a cookiecutter style
it's not an official endpoint, so it's unsurprising that it isn't super robust
could add retry logic to the modal.Function calls or start using the official API -- which is both painful and requires an API key
How to connect discord question & answers with Github issues?
from us:
bring in videos from elsewhere? need to make sure to have a fallback for chapterless videos
and fall back to our current strategy if the chapters aren't available
chapters provide better metadata for chunks, see #16
use message IDs (plus other metadata if needed) as identifiers/keys so that feedbacks can be matched
return emoji counts
this blocks #16
jank implementation of bulk feedback ingestion in a private Colab here
could probably be done by adjusting the Discord bot a bit -- it should be able to respond to reaction events, which could even be cleaner
Atlas Mongo is a Pulumi provider, so it's at least possible
should only happen after #7
right now, we assume folks have a key pair called fsdl-webserver-keys, which is unfair
unfortunately, you cannot provision ec2 keyPairs directly from pulumi-aws -- you need to generate a key pair first and then register it
ideally, we would literally provision the bot for people with Pulumi -- but it's probably easier to get them to copy-paste info into a CLI
This is a common question for chatbots, and the current prompt doesn't set the model up to give a good answer.
right now, we just point folks to the Python SDK's docs, but it's an involved process
The response to this message, which asked what is the size of gpt-4
, timed out. The Modal logs indicate that it was a 500 due to an InvalidRequestError
from exceeding the context length:
While we could play with chunk length and number of retrieved chunks to try and prevent this from happening, I think it's better to make the max_tokens
parameters for the completion requests adaptive up to a minimum length and truncating the sources if they are too long.
much cleaner and better demonstrates the "isolated environment" features of Modal
the CLI will look weird in a notebook, probably, but we can work around that
would be great if it can run off of a file or a Python object, but I'd settle for Python object
get_documents
would then call this function
at first, I thought this was not possible because bots needed to be "always on" to respond to events, which didn't jive with Modal's serverless style, so I put it on EC2, but Modal's web endpoints seem to serve as webhooks, which should work
this would simplify the automation a lot!
but even if we can do everything on Modal, it's nice to have trad cloud infrastructure in the repo, even if only as an option among others.
the transcripts are autogenerated by YouTube. an afternoon of annotation -- with either Descript or Whisper + GPT-4's help -- would make them much cleaner and that makes them more useful to folks watching the videos as well.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.