hwchase17 / langchainjs Goto Github PK
View Code? Open in Web Editor NEW๐ฆ๐ Build context-aware reasoning applications ๐ฆ๐
Home Page: https://js.langchain.com/docs/
License: MIT License
๐ฆ๐ Build context-aware reasoning applications ๐ฆ๐
Home Page: https://js.langchain.com/docs/
License: MIT License
It looks like axios-request-adapter.js
is importing parts from axios
that aren't actually exporting from that library in the latest version (0.0.13
), causing a crash when calling out to APIs using axios:
Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/core/settle' is not defined by "exports" in /Users/carlosdp/code/weaver/node_modules/axios/package.json imported from /Users/carlosdp/code/weaver/node_modules/langchain/dist/util/axios-fetch-adapter.js
at new NodeError (node:internal/errors:371:5)
at throwExportsNotFound (node:internal/modules/esm/resolve:453:9)
at packageExportsResolve (node:internal/modules/esm/resolve:731:3)
at packageResolve (node:internal/modules/esm/resolve:913:14)
at moduleResolve (node:internal/modules/esm/resolve:978:18)
at defaultResolve (node:internal/modules/esm/resolve:1080:11)
at ESMLoader.resolve (node:internal/modules/esm/loader:530:30)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:251:18)
at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:79:40)
at link (node:internal/modules/esm/module_job:78:36) {
code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
I'll see if I have time to submit a PR for a fix for this later tonight, but downgrading to 0.0.12 is a temporary fix.
That said, I think the real fix is to eliminate the openai
library and just call the HTTP API directly with axios -- it's not immediately clear to me why an adapter should be necessary for what amounts to extremely simple HTTP requests. (Also, the whole "you need to install the openai library" exception after install is a massive anti-pattern, so would be good to eliminate that anyways!)
This should be an equals sign, not a colon.
Hey! Thanks for your work on this project. It's so exciting ๐คฉ
I'm using this lib inside https://github.com/sullivan-sean/chat-langchainjs and got the following error when importing docs:
ReferenceError: Headers is not defined
at null.createRequest (./chat-langchainjs/node_modules/langchain/src/util/axios-fetch-adapter.js:96:19)
at null.fetchAdapter (./chat-langchainjs/node_modules/langchain/src/util/axios-fetch-adapter.js:20:19)
at null.dispatchRequest (./chat-langchainjs/node_modules/axios/lib/core/dispatchRequest.js:58:10)
at Axios.request (./chat-langchainjs/node_modules/axios/lib/core/Axios.js:108:15)
at Function.wrap (./chat-langchainjs/node_modules/axios/lib/helpers/bind.js:9:15)
at null.<anonymous> (./chat-langchainjs/node_modules/openai/dist/common.js:147:22)
at null.<anonymous> (./chat-langchainjs/node_modules/openai/dist/api.js:1539:123)
Where is the Headers
definition supposed to come from?
https://github.com/hwchase17/langchainjs/blob/main/langchain/src/util/axios-fetch-adapter.js#L102
From here?
https://github.com/hwchase17/langchainjs/blob/main/langchain/src/agents/tools/requests.ts#L3
I have a chat instance defined as follows:
const chat = new ChatOpenAI({
streaming: true,
callbackManager: {
handleNewToken(token) {
console.log(token)
},
handleError(error) {
console.log("ERROR", error);
},
handleEnd(result) {
console.log("END", result);
}
},
});
It doesn't seem like handleEnd
is actually called when the stream ends.
I am creating a Raycast extension to summarize a web page, but I'm running into TypeError: text.split is not a function
. Any suggestions?
Here's my stack trace:
(summarizethis-env) โ summarizethis npm run build; npm run dev
> build
> ray build -e dist
info - entry points [src/summarizethis.tsx]
info - checked TypeScript
ready - built extension successfully
> dev
> ray develop
info - entry points [src/summarizethis.tsx]
ready - built extension successfully
10:26:52.444 { url: 'https://dagster.io/blog/chatgpt-langchain' }
10:26:52.444 ===LOADER===
10:26:52.444 {
loader: CheerioWebBaseLoader2 {
webPath: 'https://dagster.io/blog/chatgpt-langchain'
}
}
10:26:52.839 ===DOCUMENTS===
10:26:52.840 {
documents: [
Document {
pageContent: `Did you miss our Watch the Airflow โ Dagster migration event? - Watch it here!PlatformCloudPricingBlogCommunityDocsSign inJoin us on Slack Star usTry Dagster CloudPlatformIntegrationsCloudPricingBlogCommunityImplementation PartnersDocsContact SalesSign inTry Dagster CloudJanuary 9, 2023 โข 13 minute readBuild a GitHub support bot with GPT3, LangChain, and Python<img alt="Pete Hunt" srcSet="/_next/image?url=%2Feditors%2Fpetehunt.jpg&w=48&q=75 1x, /_next/image?url=%2Feditors%2Fpetehunt.jpg&w=96&q=75 2x" src="/editors/petehunt.jpg" decoding="async" style="position:absolute;top:0;left:0;bottom:0;right:0;box-sizing:border-box;padding:0;border:none;margin:auto;display:block;width:0;height:0;min-width:100%;max-width:100%;min-height:100%;max-height:100%" class="rounded-full border border-gray-200 w-12 h-12"/>NamePete HuntHandle@floydophoneChatGPT, ever heard of it?ChatGPT came out a few months ago and blew everyonesโ minds with its abil<โฆ>
10:26:52.840 ===DOCUMENT===
10:26:52.841 {
document: Document {
pageContent: `Did you miss our Watch the Airflow โ Dagster migration event? - Watch it here!PlatformCloudPricingBlogCommunityDocsSign inJoin us on Slack Star usTry Dagster CloudPlatformIntegrationsCloudPricingBlogCommunityImplementation PartnersDocsContact SalesSign inTry Dagster CloudJanuary 9, 2023 โข 13 minute readBuild a GitHub support bot with GPT3, LangChain, and Python<img alt="Pete Hunt" srcSet="/_next/image?url=%2Feditors%2Fpetehunt.jpg&w=48&q=75 1x, /_next/image?url=%2Feditors%2Fpetehunt.jpg&w=96&q=75 2x" src="/editors/petehunt.jpg" decoding="async" style="position:absolute;top:0;left:0;bottom:0;right:0;box-sizing:border-box;padding:0;border:none;margin:auto;display:block;width:0;height:0;min-width:100%;max-width:100%;min-height:100%;max-height:100%" class="rounded-full border border-gray-200 w-12 h-12"/>NamePete HuntHandle@floydophoneChatGPT, ever heard of it?ChatGPT came out a few months ago and blew everyonesโ minds with its ability to an<โฆ>
10:26:52.841 ===SPLITTER===
10:26:52.841 {
splitter: RecursiveCharacterTextSplitter2 {
chunkSize: 10,
chunkOverlap: 1,
separators: [ '\n\n', '\n', ' ', '' ]
}
}
10:26:52.842 ===DOCS===
10:26:52.842 {
docs: [
Document { pageContent: 'Did you', metadata: [Object] },
Document { pageContent: 'miss our', metadata: [Object] },
Document { pageContent: 'Watch the', metadata: [Object] },
Document { pageContent: 'Airflow โ', metadata: [Object] },
Document { pageContent: 'โ Dagster', metadata: [Object] },
Document { pageContent: 'migration', metadata: [Object] },
Document { pageContent: 'event? -', metadata: [Object] },
Document { pageContent: '- Watch it', metadata: [Object] },
Document { pageContent: 'here!Plat', metadata: [Object] },
Document { pageContent: 'tformClou', metadata: [Object] },
Document { pageContent: 'udPricing', metadata: [Object] },
Document { pageContent: 'gBlogComm', metadata: [Object] },
Document { pageContent: 'munityDoc', metadata: [Object] },
Document { pageContent: 'csSign', metadata: [Object] },
Document { pageContent: 'inJoin us', metadata: [Object] },
Document { pageContent: 'on Slack', metadata: [Object] }<โฆ>
10:26:52.842 ===COMBINEDOCSCHAIN===
10:26:52.843 {
combineDocsChain: MapReduceDocumentsChain {
memory: undefined,
llmChain: LLMChain {
memory: undefined,
prompt: [PromptTemplate],
llm: [OpenAI2],
outputKey: 'text'
},
inputKey: 'input_documents',
outputKey: 'output_text',
documentVariableName: 'text',
maxTokens: 3000,
maxIterations: 10,
combineDocumentChain: StuffDocumentsChain {
memory: undefined,
llmChain: [LLMChain],
inputKey: 'input_documents',
outputKey: 'output_text',
documentVariableName: 'text'
}
}
}
10:26:52.843 ===CHAIN===
10:26:52.843 {
chain: AnalyzeDocumentChain2 {
memory: undefined,
inputKey: 'input_document',
outputKey: 'output_text',
combineDocumentsChain: MapReduceDocumentsChain {
memory: undefined,
llmChain: [LLMChain],
inputKey: 'input_documents',
outputKey: 'output_text',
documentVariableName: 'text',
maxTokens: 3000,
maxIterations: 10,
combineDocumentChain: [StuffDocumentsChain]
},
textSplitter: RecursiveCharacterTextSplitter2 {
chunkSize: 1000,
chunkOverlap: 200,
separators: [Array]
}
}
}
10:26:52.843 TypeError: text.split is not a function
at RecursiveCharacterTextSplitter2.splitText (/Users/skurilyak/.config/raycast/extensions/summarizethis/summarizethis.js:195:25)
at RecursiveCharacterTextSplitter2.createDocuments (/Users/skurilyak/.config/raycast/extensions/summarizethis/summarizethis.js:95:36)
at AnalyzeDocumentChain2._call (/Users/skurilyak/.config/raycast/extensions/summarizethis/summarizethis.js:43287:47)
at AnalyzeDocumentChain2.call (/Users/skurilyak/.config/raycast/extensions/summarizethis/summarizethis.js:16471:35)
at Object.onSubmit (/Users/skurilyak/.config/raycast/extensions/summarizethis/summarizethis.js:49351:31)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async onAction (/Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/node_modules/@raycast/api/index.js:18:5481)
Unless I'm doing something wrong, it looks like connecting to an OpenAI model through Azure is not supported yet. I've tried to add my deployed Azure model name as a parameter called "deployment_id" (as that works in the Python version of LangChain) but I'm getting an "Incorrect API key provided" error in this version. I also tried "deploymentId", "deploymentID", and "engine" as parameter names.
Are there any plans to support Azure versions of OpenAI models?
Currently the agent test is failing because of an issue with the serpapi package and jest, specifically inside of a package inside of another package (jsdom -> jsdomexceptions).
It's possible that it's a tsconfig thing but I couldn't find any information to suggest it was anything but a bad line of code or something that isn't well supported in node.
Solution ideas:
Write a script that edits the package to fix the error. Not great, but fast and works.
Switch to serp
npm package
work with the developer on serpapi to get it resolved?
Thank you for this library.
Right now it's difficult to figure out what is missing. It would be nice to have a list of things from the Python version that are not implemented.
It would also be nice to have a roadmap to communicate what is being worked on.
The Python library has a callback manager that allows setting multiple callbacks on the same instance, such as on_chain_start
, on_tool_start
, etc:
I noticed that for the LLM, there is already something similar implemented:
const model = new OpenAI({
temperature: 0,
openAIApiKey: process.env.OPENAI_API_KEY,
callbackManager: {
handleStart(llm, prompts, verbose) {
logger.info('Got handleStart', { llm, prompts, verbose })
},
handleEnd(output, verbose) {
logger.info('Got handleEnd', { output, verbose })
},
handleError(error, verbose) {
logger.error('Got handleError', { error, verbose })
},
},
})
I wonder if we could use a more event-based approach instead of callbacks. I've been playing around with this, and here is a possible way to do this:
const eventManager = new EventManager()
ZeroShotAgent.validateTools(tools)
const prompt = ZeroShotAgent.createPrompt(tools)
const llmChain = new LLMChain({
prompt,
llm: model,
})
llmChain.setEventManager(eventManager)
// ...
eventManager.on('chain:start', (data) => {
logger.info('Got event chain:start', { data })
})
eventManager.on('chain:end', (data) => {
logger.info('Got event chain:end', { data })
})
eventManager.on('chain:error', (data) => {
logger.info('Got event chain:error', { data })
})
Implementation could be something similar to this:
// ...
export abstract class BaseChain implements ChainInputs {
// ...
eventManager: EventManager | null = null
// ...
setEventManager(eventManager: EventManager) {
this.eventManager = eventManager
}
// ...
async call(values: ChainValues): Promise<ChainValues> {
const fullValues = deepcopy(values)
if (!(this.memory == null)) {
const newValues = await this.memory.loadMemoryVariables(values)
for (const [key, value] of Object.entries(newValues)) {
fullValues[key] = value
}
}
this.eventManager?.emit('chain:start', {
name: this._chainType(),
inputs: fullValues,
})
try {
const outputValues = this._call(fullValues)
if (!(this.memory == null)) {
await this.memory.saveContext(values, outputValues)
}
this.eventManager?.emit('chain:end', {
outputs: outputValues,
})
return outputValues
} catch (error) {
this.eventManager?.emit('chain:error', {
error,
})
throw error
}
}
}
Then the EventManager
could be just a simple class extending the node.js' EventEmitter, such as:
import EventEmitter from 'node:events'
export class EventManager extends EventEmitter {
override on(
eventName: 'chain:start',
listener: (data: { name: string, inputs: Record<string, string> }) => void
): this
// ... other overrides to make this type safe...
override on(eventName: string, listener: (...args: any[]) => void): this {
return super.on(eventName, listener)
}
}
I got this mostly working locally by overriding the LangChain classes, so I can open a PR if needed.
Use this in axios adapter https://github.com/Azure/fetch-event-source
@sullivan-sean what do you think? The current setup makes the tsconfig "invalid" after yarn build
is run, because of the generated top-level entrypoints, if we could scope tsconfig to src/
would be easier?
This is to be able to easily implement remote caches, eg. redis
See https://discord.com/channels/1038097195422978059/1076182741374214285/1083153092658331658
At the very least we should support extracting from multiple keys (JSONPointers) and concatenate them (with a newline to separate)
But to support the format in the message we'd have to do a little more
Is there any way to see the intermediate steps in the mapreduce chain when using the OpenAI
llm?
I'm only able to see the original prompts at this time, even when setting it to "verbose"
Is there a way to preview the last output prompt before actually calling the LLM?
Right now there is a reliance on fs
and other native node modules which do not exist in the Vercel edge runtime. It would be amazing if we could publish a langchain/edge
with the functionality that can be run at edge
Related to: langchain-ai/langchain#1260
The default embeddings (e.g. Ada-002 from OpenAI, etc) are great generalists. However, they are not tailored for your specific use-case.
Write a full-stack TypeScript application which uses custom embeddings and could support Edge Runtimes with limited features (1, 2)
๐ Customizing Embeddings!
โน๏ธ See my tutorial / lessons learned if you're interested in learning more, step-by-step, with screenshots and tips.
flowchart LR
subgraph "Similarity Search"
direction LR
CustomMatrix["Custom Embedding Matrix\n(e.g. custom-embedding.npy)"]
Multiply["(Original Embedding) x (Matrix)"]
CustomMatrix --> Multiply
Text1["Original Texts #1, #2, #3..."]
Raw1'["Original Embeddings #1, #2, #3, ..."]
Custom1["Custom Embeddings #1, #2, #3, ..."]
Text1-->Raw1'
Raw1' --> Multiply
Multiply --> Custom1
DB["Vector Database"]
Custom1 -->|Upsert| DB
Search["Search Query"]
EmbedSearch["Original Embedding for Search Query"]
CustomEmbedSearch["Custom Embedding for Search Query"]
Search-->EmbedSearch
EmbedSearch-->Multiply
Multiply-->CustomEmbedSearch
SimilarFound["Similar Embeddings Found"]
CustomEmbedSearch -->|Search| DB
DB-->|Search Results|SimilarFound
end
.npy
file.import { CustomizeEmbeddings, OpenAIEmbeddings } from "langchain/embeddings";
/* === Generalized Embeddings === */
/* Embed queries */
const embeddings = new OpenAIEmbeddings();
const res = await embeddings.embedQuery("Hello world");
/* Embed documents */
const documentRes = await embeddings.embedDocuments(["Hello world", "Bye bye"]);
/* === Training Customized Embeddings === */
// Not supported in JavaScript/TypeScript. Use Langchain for Python: https://github.com/hwchase17/langchain/issues/1260
/* === Loading Customized Embeddings === */
const customEmbeddings = new CustomizeEmbeddings(embeddings);
customEmbeddings.load("custom-embedding.npy")
/* Embed queries */
const customRes = await customEmbeddings.embedQuery("Hello world");
/* Embed documents */
const customDocumentRes = await customEmbeddings.embedDocuments(["Hello world", "Bye bye"]);
P.S. I'd love to personally contribute this to the Langchain repo and community! Please let me know if you think it is a valuable idea and any feedback on the proposed solution. Thank you!
So far it seems like there are 3 culprits which may block us from running Langchainjs
on Edge.
1) OpenAI and it's usage of axios
#118
2) **/index.ts
files might include other modules, which aren't needed
3) langchain/util/hub
and the general usage of NodeJS APIs (fs
, etc.) #97
4) Have at least 1 vector store which 1. doesn't need Node native modules and 2. stores metadata for each document #73
Add tag in the docs for node-only things (eg hnswlib)
Create and test cloudlfare workers and vercel edge templates
Remove usage of node-fetch (which uses node-only APIs) #118
Remove all usage of require #124
Get TS to output ESM #124
Remove usage of node crypto
#213
Replace node-only tokeniser with one that works in browser-like envs #214
This is the error output when attempting to deploy the examples/src/chains/llm_chain.ts
via Vercel:
https://github.com/replicate/replicate-js simple way to call open source models from JS
Tried to run:
/* Split text into chunks */
const textSplitter = new RecursiveCharacterTextSplitter({
chunkSize: 1000,
chunkOverlap: 200,
});
Also had to add legacy-peer-deps=true to my .npmrc because I am using "@pinecone-database/pinecone": "^0.0.9" and it wants 0.0.8.
Hi,
I'm trying to figure out how to use LangChain in a VisualStudio Code extension. I'm aware that the lib currently doesn't support this environment, but maybe someone could give some pointers and I could hopefully create a PR?
Got this error whenever I try to import OpenAIChat
:
Error: ENOENT: no such file or directory, open '/vscode-chatgpt/out/encoder.json'
at Object.openSync (/vscode-chatgpt/lib/fs.js:585:3)
at Object.func [as openSync] (/vscode-chatgpt/lib/electron/js2c/asar_bundle.js:5:1812)
at Object.readFileSync (/vscode-chatgpt/lib/fs.js:453:35)
at Object.e.readFileSync (/vscode-chatgpt/lib/electron/js2c/asar_bundle.js:5:9160)
at node_modules/gpt-3-encoder/Encoder.js (/vscode-chatgpt/node_modules/gpt-3-encoder/Encoder.js:5:31)
at __require (/vscode-chatgpt/out/extension.js:12:50)
at node_modules/gpt-3-encoder/index.js (/vscode-chatgpt/node_modules/gpt-3-encoder/index.js:1:28)
at __require (/vscode-chatgpt/out/extension.js:12:50)
at node_modules/langchain/dist/llms/base.js (/vscode-chatgpt/node_modules/langchain/src/llms/base.ts:1:24)
at init_base (/vscode-chatgpt/out/extension.js:9:56)
at node_modules/langchain/dist/llms/index.js (/vscode-chatgpt/node_modules/langchain/src/llms/index.ts:1:1)
at __init (/vscode-chatgpt/out/extension.js:9:56)
at node_modules/langchain/llms.js (/vscode-chatgpt/node_modules/langchain/llms.js:1:1)
at __init (/vscode-chatgpt/out/extension.js:9:56)
at /vscode-chatgpt/src/chatgpt-view-provider.ts:142:33
at ChatGptViewProvider.sendApiRequest (/vscode-chatgpt/src/chatgpt-view-provider.ts:142:27) {errno: -2, syscall: 'open', code: 'ENOENT', path: '/vscode-chatgpt/out/encoder.json', stack: 'Error: ENOENT: no such file or directory, opeโฆ-chatgpt/src/chatgpt-view-provider.ts:142:27)', โฆ}
rejected promise not handled within 1 second: Error: ENOENT: no such file or directory, open '/vscode-chatgpt/out/encoder.json'
stack trace: Error: ENOENT: no such file or directory, open '/vscode-chatgpt/out/encoder.json'
at Object.openSync (node:fs:585:3)
at Object.func [as openSync] (node:electron/js2c/asar_bundle:5:1812)
at Object.readFileSync (node:fs:453:35)
at Object.e.readFileSync (node:electron/js2c/asar_bundle:5:9160)
at node_modules/gpt-3-encoder/Encoder.js (/vscode-chatgpt/node_modules/gpt-3-encoder/Encoder.js:5:31)
at __require (/vscode-chatgpt/out/extension.js:12:50)
at node_modules/gpt-3-encoder/index.js (/vscode-chatgpt/node_modules/gpt-3-encoder/index.js:1:28)
at __require (/vscode-chatgpt/out/extension.js:12:50)
at node_modules/langchain/dist/llms/base.js (/vscode-chatgpt/node_modules/langchain/src/llms/base.ts:1:24)
at init_base (/vscode-chatgpt/out/extension.js:9:56)
at node_modules/langchain/dist/llms/index.js (/vscode-chatgpt/node_modules/langchain/src/llms/index.ts:1:1)
at __init (/vscode-chatgpt/out/extension.js:9:56)
at node_modules/langchain/llms.js (/vscode-chatgpt/node_modules/langchain/llms.js:1:1)
at __init (/vscode-chatgpt/out/extension.js:9:56)
at /vscode-chatgpt/src/chatgpt-view-provider.ts:142:33
at ChatGptViewProvider.sendApiRequest (/vscode-chatgpt/src/chatgpt-view-provider.ts:142:27)
Here's my tsconfig.json
:
{
"compilerOptions": {
"module": "NodeNext",
"target": "ES2020",
"lib": [
"ES2020"
],
"sourceMap": true,
"rootDir": "src",
"strict": true /* enable all strict type-checking options */
}
}
I'm using dynamic imports as the extension uses commonJS:
const { PromptTemplate } = await import("langchain/prompts");
const { OpenAIChat } = await import("langchain/llms");
const { LLMChain } = await import("langchain");
I'm getting the following error when trying to run some of the examples locally.
Error: Package subpath './llms' is not defined by "exports" in /Users/jameszammit/Github/stuff/llm/node_modules/langchain/package.json
Running this code:
import { initializeAgentExecutor } from "langchain/agents";
gives me this error:
Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/Users/USERXXX/Projects/langchain/prototype/javascript/node_modules/langchain/agents' imported from /Users/USERXXX/Projects/langchain/prototype/javascript/prototype.js
Did you mean to import langchain/prototype/javascript/node_modules/langchain/agents.js?
When I add the .js
to the import (like this: import { initializeAgentExecutor } from "langchain/agents.js";
) it works fine. This same issue occurs for all imports I've tried. It's been a while since I worked with javascript so maybe I'm missing something in the es6 module import process that's obvious but wondering if I'm doing something wrong here or whether the documentation should be updated. I assume it should work without the .js
suffix.
Running node v18.13.0 on macOS Catalina 10.15.
It seems that this is a known issue but wanted to call it out explicitly since we're (Diagram) interested in helping bring the WASM tokenizer up to parity with python tiktoken for our usage. Not currently using langchain, but would be interested in adopting bits and pieces.
While trying out Chroma integration I stumbled upon the following error:
Error: Please install chromadb as a dependency with, e.g. `npm install -S chromadb`
Even though I had already installed chromadb
.
When I dug into the source, I was surprised to see that ChromaClient
is being required and initialized directly here: https://github.com/hwchase17/langchainjs/blob/main/langchain/src/vectorstores/chroma.ts#L10-L15
This doesn't work well with cli tools like tsx.
Any reason why this is the case? Why can't we just pass our own initialized ChromaClient
to fromDocuments
? Also, this would cause an issue when ChromaDB has a different basePath
and is hosted on a remote machine as currently there is no way to configure parameters that are passed to ChromaClient
.
This is already being done with PineconeStore
where we pass in the index
initialized outside the PineconeStore
itself.
Keeping the API consistent with PineconeStore
, my suggestion would be to have this function signature for fromDocuments
(and fromTexts
too):
static fromDocuments(
client: ChromaClient,
docs: Document[],
embeddings: Embeddings,
collectionName?: string,
url?: string
): Promise<Chroma>;
And of course remove the need to initialize ChromaClient
directly and let us pass the initialized client instead.
As per the docs and latest Pinecone library, the code below should work. However, the function PineconStore.fromDocuments
throws an error as per below. It appears there is an issue passing the vectors to Pinecone.
code:
const pinecone = new PineconeClient();
await pinecone.init({
environment: `${process.env.PINECONE_ENVIRONMENT}`,
apiKey: `${process.env.PINECONE_API_KEY}`,
});
const index = pinecone.Index("langchainjsfundamentals");
// this is the cause of the error
const vectorStore = await PineconeStore.fromDocuments(
index,
docs,
new OpenAIEmbeddings()
);
Error log:
error PineconeClient: Error calling upsert: PineconeClient: Error calling upsertRaw: RequiredError: Required parameter requestParameters.upsertRequest was null or undefined when calling upsert.
Hi, I'm trying to run examples locally with this command:
yarn example prompts/few_shot.ts
However, I'm getting these errors:
Cannot find module 'langchain/prompts'
Cannot find module 'langchainjs/node_modules/langchain/prompts.js'
But I have ran yarn install in top-level project so I'm not sure if I'm missing another step. I don't have too much expereince with mono-repos so any help is much appreciated.
something like:
const model = new OpenAI(
{},
{
...HeliconeOptions({
propertySession: "24",
propertyConversation: "support_issue_2",
propertyApp: "mobile",
cacheEnabled: false
}),
otherAxiosConfigurationOption: "value"
}
)
There are few things that can be improved in the integration:
$contains
a string. https://github.com/chroma-core/chroma/blob/main/examples/where_filtering.ipynbHi there,
I'm running through the docs and I'm essentially stuck here: https://hwchase17.github.io/langchainjs/docs/getting-started#agents-dynamically-call-chains-based-on-user-input
This makes use of SerpApi which is great, but their signup process requires an email and phone number. For quickstart docs, I personally feel that a tool that doesn't require an API key (or at the very least extensive signup process) would be more beneficial
Thanks!
Typescript has extremely powerful template literals. https://www.typescriptlang.org/docs/handbook/2/template-literal-types.html.
I'd imagine one of the goals of langchainjs is to keep a similar API to the python version so you could keep your API similar but infer what the arguments are suppose to be from the string.
type Split<S extends string, D extends string> = string extends S
? string[]
: S extends ''
? []
: S extends `${infer T}${D}${infer U}`
? [T, ...Split<U, D>]
: [S];
type ExtractTemplateParams<TPath extends string> = {
[K in Split<TPath, '{{'>[number] as K extends `${infer P}}}${string}`
? P
: never]: string;
};
function createPromptTemplate<T extends string>(
prompt: T,
params: ExtractTemplateParams<T>
) {}
const prompt = 'Tell me a {{jokeType}} joke.';
createPromptTemplate(prompt, { jokeType: "test" })
createPromptTemplate(prompt, { test: 'asf' }); // <- Will throw a type error
This next example is easier to understand but is more of a departure from how python does things. It has a typed generate function that auto completes prompts and will throw an error if you pass in a prompt that isn't valid. The prompt creation helper functions also have autocomplete for their args and the possible options for those args.
https://www.typescriptlang.org/play?#code/MYewdgzgLgBA3jAwuAZgSwOYFcBOBDKNcAGhgHkAHAUzAEEBJWitGAXxgF4YcqBHLNDwAUAchDUweNCICUAbgBQC0JFgr02fIXCcYYKgHckqTLgJEwQuApgw8zANJUAngC4YFHCGBUIEAHQ0AG7+ZAAKAKIAcgwA+rRh9LEOEQCaxAqs8srg0DDiNFK6+kaUNAxMaELqploW2QpQztQwYSBUALa0OBgQughQ4mjA7iIANiBBVCIwAD4wIig4aDQAJhAAFmgUM-OLEwZUOBAzrIoqecA8BFRtnWFeHRSwXFYwg8zAbO53XT0QMk4AD4bDAAAYAdWWUCodg87Q6dgARiAsLAACRwD7DVj+MF2PoXKCKJotX4PEBPKAAFU6FDGN10ACUqFBcGBqc0qAAeUlUEAoGBXKg3cmPZ5AxSNLkwABSIAA1lRur1+jAAFaKqic6ijFBYMBgZy7BarLAdJEmkSrPCrU7nXJqa4w+VKilU3RvTVKnWw1juV3K-6Ajgg2xg2ljMYwDqwvAwTHe7Vc9hJvEEoWOkkywPu560p4MmHM1ns328rkCoXOqi58VQSVKfVgYDaMAwDA0I43ISeSnPH4IvM0ulF2HzOv9keFm6A6y2HhsnDtgqSND+YU3ZCF1kWKyg2wdECrKhjUYwgAeUAAtDagmgW2hrwAGZ8AZhEGQPMAA9D+YBskxHPkUw4B49ZUKs7wgDAEBULCUAbAhXIQN+fZUhkthZIorAKEAA
Don't think we need to specify input variables in constructor as part of BasePromptTemplateInput
-- can just be inferred from template
(First off - I fucking love LangChain. Cannot appreciate you guys enough for building & maintaining this tool!)
I've encountered issues using LangChain with the new ChatGPT API. The new API has a different behaviour than the davinchi
model. This caused the model to not follow the prompt instructions correctly and reliably.
Below was the code that I used to run a very simple Agent example
const run = async () => {
const model = new OpenAI({
modelName: 'gpt-3.5-turbo', // <-- switching back to davinchi model fixes this issue.
openAIApiKey: '...',
});
const tools = [
new DynamicTool({
name: "Addition",
description:
"call this to add two numbers together. input should be two numbers separated by a space.",
func: async (input: string) => {
const [a, b] = input.split(" ").map((x) => Number(x));
return `${a + b}`;
}
}),
new DynamicTool({
name: "BAR",
description:
"call this to get the value of bar. input should be an empty string.",
// eslint-disable-next-line @typescript-eslint/require-await
func: async () => "BAR",
}),
];
const executor = await initializeAgentExecutor(
tools,
model,
"zero-shot-react-description"
);
const input = `What is the value of BAR`;
console.log(`Executing with input: "${input}"...`);
const result = await executor.call({ input });
console.log(`Got output "${result.output}"`);
};
Output:
error - Error: Could not parse LLM output: That was straightforward. I now know the value of BAR.
I added a console.log()
to the extractToolAndInput()
function to see the ChatGPT output:
BAR is a constant, so I just need to call it with the BAR function.
Action: BAR
Action Input: ""
That was straightforward. I now know the value of BAR.
We expected the final answer to include Final Answer: BAR
, but it did not. ChatGPT failed to follow the current prompt instruction.
From the new ChatGPT completion doc, see this example:
import openai
openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who won the world series in 2020?"},
{"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
{"role": "user", "content": "Where was it played?"}
]
)
Posting this as hopefully a way for us to discuss how to move forward with the new ChatGPT model. More than happy to open a PR and contribute! Let me know how I can help :^)
Hey! Why isn't this available on npm yet?
First of all, I'm glad to use langchain in Javascript. Thank you to the creators.
However when I tried to use langchain.js in React-vite project, the code seems to fail.
This code is React-vite initial code with langchain's LLM import.
import { useState } from "react";
import reactLogo from "./assets/react.svg";
import "./App.css";
import { OpenAI } from "langchain";
function App() {
const [count, setCount] = useState(0);
const llm = new OpenAI({ temperature: 0.9 });
return (
<div className="App">
<div>
<a href="https://vitejs.dev" target="_blank">
<img src="/vite.svg" className="logo" alt="Vite logo" />
</a>
<a href="https://reactjs.org" target="_blank">
<img src={reactLogo} className="logo react" alt="React logo" />
</a>
</div>
<h1>Vite + React</h1>
<div className="card">
<button onClick={() => setCount((count) => count + 1)}>
count is {count}
</button>
<p>
Edit <code>src/App.tsx</code> and save to test HMR
</p>
</div>
<p className="read-the-docs">
Click on the Vite and React logos to learn more
</p>
</div>
);
}
export default App;
Run this app yarn dev
then, I got this error in console.
Uncaught SyntaxError: The requested module '/node_modules/.vite/deps/langchain.js?v=74bcca3d' does not provide an export named 'OpenAI' (at App.tsx:4:10)
Not only OpenAI, other classes also give me import error.
Running node v16.13.0 on macOs Montery 12.3.1 , apple silicon m1 chip.
Anyone know how to resolve this probrem?
We are getting
Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: No "exports" main defined in langchain\package.json
It started happening from versions 0.0.13 and up.
It works fine for 0.0.12.
equivalent of https://langchain.readthedocs.io/en/latest/modules/chains/examples/sqlite.html
a chain that generates sql code, then executes it, then summarizes responses
prereq for this is probably a sql tool + example of how to use it (separate pr)
https://github.com/tantaraio/voy
A WASM vector store written in Rust, so a good choice for non-Node envs
Hi, already flagged that (and added my solution) on Discord - but felt like having it on GitHub might be a bit more productive.
Providing documents with not enough or too much content to HSNWLib returns an error:
Invalid the number of k-nearest neighbors (cannot be given a value greater than `maxElements`: 3).
My current solution has been to add this condition under line 76 in hsnwlib.js :
if(k > this.index.getMaxElements()) k = this.index.getMaxElements();
Obviously, I am not proposing this workaround as a pull request as, I believe, it is not a properly adapted long-term solution.
Hello,
I get the following error:
Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './text_splitter' is not defined by "exports" in ...node_modules/langchain/package.json
for
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
I believe it would be highly possible to use a tool such as Zod to specify the expected inputs of a tool. Because, unlike typescript types, Zod schemas are available at runtime -- the agent could also be given the stringified version of the zod schema, which would allow it to know which arguments it should produce. Zod is also a parser, so there could be a neat little iteration loop where the model could try a few times to input a configuration that would pass the schema parser.
I'd be happy to give this a crack if it sounds interesting.
Example: In my recent PR: #186
I wanted to be able to specify more than just the prompt for the image.
However, I defaulted to the easier mode because parsing out from a possibly quite wildly structured string could be an issue.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.