paradigmxyz/flux

Graph-based LLM power tool for exploring many completions in parallel.

flux's Issues

feat: Local Model Compatibility

Is your feature request related to a problem? Please describe.
Limiting this to OpenAI is severely hampering its capabilities

Describe the solution you'd like
A field to connect this to different endpoints for running different, possibly multiple, LLMs. I think it is reasonable to assume their output is similar in format to OpenAI's API.

feat: Whisper speech-to-text

Is your feature request related to a problem? Please describe.
It would be helpful to add whisper support to allow users to transcribe their speech to text, as some users may prefer this option.

Describe the solution you'd like
A microphone button (and maybe a keybind) that will allow users to utilize speech-to-text when prompting LLMs. Still researching different ways to accomplish this but it should be relatively straightforward to implement.

Describe alternatives you've considered
Whisper makes the most sense because you can use the same API key. Other options can be researched later, possibly when adding #71 support.

Additional context
You can go ahead and assign this to me @transmissions11 just opening it for tracking purposes. I'll also open a draft PR once I start coding so it's visible early on.

feat: make horizontal panning easier for mouse users

Is your feature request related to a problem? Please describe.
When utilizing a keyboard and mouse, there isn't an intuitive way to pan horizontally, unless I'm missing something. Feels great on a trackpad but you don't have that freedom with a mouse currently.

Describe the solution you'd like
Change panActivationKeyCode={null} to panActivationKeyCode="Space" within the <ReactFlow /> component to allow users to click and hold Spacebar + drag the mouse to pan around the screen. The prop can also be removed if preferred since the default for this behavior is Space.

Happy to contribute this change myself once a decision is made!

Describe alternatives you've considered
Open to other Keybinds, suggestions, or to be educated on the design decision (or the mouse pan method I'm missing). panActivationKeyCode="Shift" + Drag is another viable option, it doesn't seem to override the multi-select behavior.

Additional context
https://reactflow.dev/docs/guides/panning-and-zooming/

feat: selecting multiple nodes with `Shift` button

When #7 is implemented, it's clear that not all of the nodes are effective enough to be exported. Some of them should just go to the void.

Currently, you can only select with the mouse.

feat: get responses from model at multiple temps from a single generation

similar to #14

should just be a menu bar option

feat: dark mode toggle

Is your feature request related to a problem? Please describe.
yes i code in the dark sometimes

Describe the solution you'd like
dark mode

Describe alternatives you've considered
some dark mode chrome extension

Additional context
it's bright

I think someone is trying to copy your project without giving credits: https://github.com/adithya-s-k/TOT-AGI/issues/1

Check out: https://tot-agi.vercel.app/

Is it a copy of flux?

feat: workspaces

What if I as a GPT power user never want to leave flux?

Abstract

Workspaces would enable users to manage multiple Conversation Trees in one organized space, to easily navigate between them.

Problem

Project-specific AI tasks:

1. I generate ideas –> I want to add some degen input from GPT –> Creating a tree for Idea Iterations & Enhancing;
2. I manage projects –> I want GPT to write the issues for me, following my guidelines –> Creating a tree for GitHub Project Management;
3. I have a specific stack of frameworks & tools I build with –> I want GPT to help me with writing boilerplate and fixing my stuff (davinci is better for such things, but hey GPT is multimodel) –> Creating a tree for Code Helper

Some can have a shared context with a connection to a Parent Node, or not.

There could be a code-specific context that is intended to be shared across some of the projects, but with such a setup, things would get messy as I as a user would need to move those nodes around to compose them in "groups" at least visually.

Specification

• There can be $1...W$ workspaces;
• Workspace can hold $1...N$ nodes;
• Workspace does not require nodes children or parent to be in the workspace too;
• Visually represented as a circle-edged rectangular with dash-stroke.
• Navigation between them is done via Shift+Cmd+<Left Arrow> and Shift+Cmd+<Right Arrow>.
• Workspace can have a name (cc #8)

fix: ensure keybinds work properly on windows

currently keybinds don't really work well on windows:

• we use the meta key when i think the standard is the ctrl key (need to double check on this)

• we always display the command key when showing keybinds

would be great to fix this for our windows frens

feat: track more events with Mixpanel

Is your feature request related to a problem? Please describe.
Right now we have Mixpanel for tracking specific user events but we don't track that much.

Describe the solution you'd like
Add more tracking so we can improve UI/UX based on actual usage.

feat: persist size of right chat

Is your feature request related to a problem? Please describe.
It would be nice to persist the size of right chat so when I close my window and come back I still have the same size of chat that I had.

Describe the solution you'd like
When I reload page my right chat has the same size.

Additional context
We can use localStorage to persist the size.

feat: auto label nodes w/ the first X chars of their content

Is your feature request related to a problem? Please describe.
Users find it difficult to navigate large trees without clear labels. Users have become accustomed to their conversations being essentially auto-labeled in the ChatGPT UI. Flux conversations by comparison are opaque and require more mental memory.

Describe the solution you'd like

Take the first X or so characters (ideally snapping to the nearest full word less than that limit) and make that the node's name, unless it has a custom label

Describe alternatives you've considered

We could try to use ChatGPT/etc to auto label conversations like OpenAI does in the ChatGPT ui, but I don't this will be effective on a per node basis and will be too expensive for large trees.

feat: window.ai support

Describe the solution you'd like
Allow users to log in via https://windowai.io/

If they have the extension installed, just use it and don't prompt for API key or show the input, otherwise work as normal.

bug: openAI key modal is not dismissible even after pasting the token

No matter what I do ,the modal won't go away.

I refresh, the token is loaded from localhost. but the model still is not going away.

When I remove the HTML via devtools the app seems to work fine.

fix: regenerating GPT nodes that are actively generating should cancel the previous stream

as to prevent gibberish situations like this occurring: https://www.loom.com/share/0efcd2bea3a840359467fa175ea4cb9d

feat: get responses from multiple models from a single generation

Hi,

I very often do response comparisons between GPT 4 and GPT 3.5. I would love an option like (nat.dev), to generate one leaf from GPT4, another from GPT3.5. It would also be cool to be able to change those in middle of the route to test efficient chaining (ex: cleanup using 3.5, important classification using 4 etc)

feat: collaborative mode

Is your feature request related to a problem? Please describe.
It would be useful to collaborate with someone on the same page, since flux is getting more structurization features with #8 and possibly #18.
Describe the solution you'd like
A collaborative mode can be implemented with a button view, which on click generates a unique link and opens a non-closable modal on the corner of a screen with a link that the user can copy to send to anyone to collaborate on the prompts.

Something similar to what codesandbox has:

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context

• https://reactflow.dev/docs/examples/interaction/collaborative/

MIXPANEL_TOKEN is not def ined at main.tsx:18:12

Per twitter thread, this should prob. be only on prod?

feat: Chat Search

Is your feature request related to a problem? Please describe.
Be able to search in flux. After a couple of levels there can be a lot of chats the ability to search through every chat would make it easier to find what you're looking for

Describe the solution you'd like
Something like an algolia search in react.dev, and maybe even a keyboard short cut to bring the search bar up

Describe alternatives you've considered
elasticsearch can also be used instead of algolia

feat: allow providing custom names to the message

Per OpenAi cookbook it's possible to give messages custom names, and those are somehow accessible to the model?

Would be great if naming nodes can be reflected in the UI (similar to #8 ) but if it's visible to the models, not sure I'd suggest to do it the other way around (ie, user replaces the node name for visual convinience #8 it shouldn't affect his evaluations )

feat: Add ReactFlow `<Controls/>` and link to github

My mouse scroll-wheel is broken on my desktop so I can't zoom in and out of the canvas. If I could use the zoom in and out buttons provided by ReactFlow <Controls/> then I'd be able to.

Also I wanted to give feedback/contribute to the project but the Github link was not readily available in the app itself, so I needed to go through a twitter thread or do a google search.

feat: ability to change the type of node?

Would save 3 clicks if I could change the node type from system to user and vise-versa (so I won't have to reconnect the nodes after preparing the tree)

feat: partial tree export/import

Allow users to share the state of the agent, "tree import/export".

Use case
User A prompts several times to get to a state where the agent is fairly good at doing X type job.
User A shares the agent with their friend, User B.
User B uses the agent regularly to do X type job.
User B discovers a better state for the agent and shares it with User A.

feat: make regeneration not override nodes by default

can add a new keybind to do overriding

https://twitter.com/spring_stream/status/1640824513213652992?s=20

chore: automate PR descriptions with PR-Codex Github app

Is your feature request related to a problem? Please describe.

Suggesting to automate PR descriptions with ChatGPT, for quicker reviews

Describe the solution you'd like

Install PR-Codex Github app in the repo, to summarize PR code diffs in Github discussion. More info on how it works on the website and repo

fix: while responses are being generated, copy code buttons become difficult to click

Platform
OSX Chrome

Description
While a response is being streamed in, if there is a codeblock with a copy code button, clicking on it will seem to do nothing or require many clicks.

I believe this is because the button is being re-rendered on every new token? Regardless, should always be easy to click the button whether text is generating or not.

fix: better highlight to show selected node

this red highlight does a great job of making it extremely clear which node you've selected which is helpful for navigation

but its ugly and doesnt match the rest of our color scheme

open to suggestions on a design that looks decent but still makes it super clear which node you've selected

fix: renamed nodes are being auto-renamed once again

Platform
MacOS, Google Chrome
Description
When I rename the node by myself and then edit its body, my node is being renamed once again.

As I filled the node name by hands, I would like the node to be named as I originally named it.

Affected-by #33

feat: editing response textboxes should stop GPT generation

feat: scroll down and down when generating response

Is your feature request related to a problem? Please describe.
It would be nice if right will constantly move down when generating response that doesn't fit in my viewport.

Describe the solution you'd like
As ChatGPT has the functionality of scrolling down when generating but also I can scroll up and won't be move down again and again.

Describe alternatives you've considered
Arrow down icon button that will keep up with generated text.

fix: Text garbling on long responses

Platform
Linux - Brave

Description

Unsure of the precise cause of this one, but this has occurred a few times on long, code-block responses in the past few days. Using GPT-4, with certain long responses the text starts to become garbled. It feels like this is a sampling issue, or the token ids are getting mis-attributed.

I've performed an identical query from the OpenAI playground right after with zero issues, so I suspect this is a Flux specific bug. Will keep an eye out for a reproduce-able query, but thought it was worth lodging in the tracker.

Ctrl-backspace removes node instead of the word in the input

Platform
Linux(pop_os) / gnome, Opera

Description
Web app intercepts ctrl-bkspace and handles it as "delete node" instead of deleting word to the left of the cursor. It's a default behaviour across the system for any text input (I believe in Windows it works the same)

Related to #40

I expected to see this happen: the word to the left of cursor should be removed

Instead, this happened: the node got deleted

---

(sidenode: thanks for open sourcing this project!)

perf: github plugin that shows bundle size diff on every PR

self explanatory from title — would be nice to have exact numbers/percentages on how much each PR and the deps they add contribute to bundle size to avoid bloating the app

feat: node color customize

Thanks for this repo. It's quite interesting ❤️

Is your feature request related to a problem? Please describe.
#8 rename node is an essential feature. Another feature related is node customize.

Describe the solution you'd like
Customize is a bit complex, but I believe node color customize is a great feature to have, user can categorize the nodes visually by their color.

feat: change keybinding for remove node from command + backspace

Is your feature request related to a problem? Please describe.
Yes, as a mac user I often use command + backspace to delete the current line.

Describe the solution you'd like
Change from command + backspace to something equally ergonomic. I'm new to the tool, so I would prefer whatever is pragmatic in the context of the other keybindings.

Describe alternatives you've considered
None

Additional context
None

feat: keybind to jump between trees

Is your feature request related to a problem? Please describe.
We have keybinds to jump between parent, child, and sibling nodes, but none to jump between trees, which I often find myself wishing to do.

Describe the solution you'd like
Add keybinds Command + Shift + Left and Command + Shift + Right to navigate to trees in the respective directions, like how navigation between sibling nodes works.

Describe alternatives you've considered
This is basically a subset of the workspace feature requested in #18. That feature is a bit larger and I think we still need a bit more time to decide if that's the optimal way to go, but this specific sub-feature is very tractable and useful even in isolation.

feat: Automatically create a new node/tree when editing to preserve history

Is your feature request related to a problem? Please describe.
I've been using flux for a bit and just realized that when I edit a node in the "middle" of a tree, it overwrites the past version which generated its downstream nodes, so the chat view is inconsistent when looking from those downstream nodes, because the prompt that generated the response is different. I know that if I ctrl+z the history is still there, but I'm concerned that if I export the chat by copying to the clipboard, or via new functionality in #67, then it will be inaccurate.

Describe the solution you'd like
I'd like for edits to a node to simply duplicate the node, and make it a sibling of the original version. Then further paths can be explored without overwriting old ones.

Describe alternatives you've considered
This could be a configurable option in the settings modal.

perf: useMemo/useCallback/etc

There's currently a lot of areas of the code where callbacks and objects could be memoized but are not. Some help here would be appreciated.

Only thing to note is the fact that nodes and edges (which are both objects) are at the core of everything makes things a little difficult I believe — as we'd need to do a deepequals check in some places which might end up being even more expensive?

fix: shift selection works on click instead of holding

Platform
MacOS, Google Chrome.

Description
⇧ Shift key selects multiple nodes not by holding a key, but just by clicking.

Screen.Recording.2023-04-02.at.13.11.45.mov

feat: typo patcher keybind

often i'll be rushing to jot something down in a user node and have a text with a bunch of typos, etc

having a single keybind that just calls the selected LLM with a prompt like "Correct any typos in this text: {SELECTED_NODE_CONTENTS}" and then replaces the contents of the node w/ the output would be sick

fix: bullet points with a space in-between them render weirdly

Platform
Chrome — OSX

Description

The following markdown:

**Flux** is really _cool_

- test

- test

1. hi

2. hi

Renders like so:

(see how the contents of the bullets are on a separate line?)

This only seems to happen when there is a full blank line between bullets

bug: shouldn't allow connecting multiple parents to the same node

was playing around with connecting random things and got this
which does nothing (I think?)

feat: max-tokens for completion

should add a setting to constrain the max # of tokens in completions

feat: auto-layout

Is your feature request related to a problem? Please describe.
When you create a new node, it usually is created at the right of the last child node, which breaks the visual appearance of a tree.

Describe the solution you'd like
Add automatic layout positioning feature.

Additional context

• https://reactflow.dev/docs/examples/layout/auto-layout/

feat: code blocks

I think it would be beneficial to add code blocks and syntax highlighting to GPT responses. I took a stab at this already here #19

feat: markdown formatting

ChatGPT supports it, so Flux should also support it.
In difference with ChatGPT, I would enable such Markdown Formatting as a selectable option.

It is sometimes useful to ask GPT for markdown-style results to read a better-organized response, but sometimes you need the raw markdown result (in my case it helps me with GH issues).

perf: extract prompt rows into a component & memoize

when you look at the profiler for the Prompt component you'll quickly notice we re-render every row on every prompt re-render, which shouldnt be necessary w/ the right tweaks

fix: error handling if user has no access to gpt4 yet

The problem

At the moment, GPT4 rollout isn't available for everyone yet, if an API key is used that does not have access yet we should at least show something.

Error when calling model

{
    "error": {
        "message": "The model: `gpt-4` does not exist",
        "type": "invalid_request_error",
        "param": null,
        "code": "model_not_found"
    }
}

Multiple ways to fix:

1. We check if the error is error.code = model_not_found and display a message telling the user to switch because they don't have access yet. But this requires us to implement some general error
2. When supplying an API key we call https://api.openai.com/v1/models which returns the available models and we lock GPT4 if it isn't in this list and lock it if is.
3. When switching models we call https://api.openai.com/v1/models to see if this key has access to GPT4 and lock the model switch in settings.

2 and 3 can maybe be done in combination because if a user gets access while they already had supplied the key on Flux they would have to switch api keys for it to refetch this information.

feat: rename nodes

https://twitter.com/nerdymomocat/status/1640850983097077762?s=20

feat: bundle as a tauri application

I thought it may be cool to also bundle the app as a tauri application.

Especially when it lives in you menubar:

This could come very handy when using it as a permanent companion like I do.
Please see my branch for a demo.