Comments (9)
Yes. I would even recommend trimming the story by paragraph instead of sentence, word or token, getting rid of the oldest first.
from novelai-api.
Thank you for your response on this. I thought I could revive this thread with some follow up.
So then it is true, that the API itself doesn't know anything about Memory, Authors Note, etc. It's just the WebUI placing these text snippets at a specific position in the prompt, which is only one big string.
try to remove the first lines of Story until Memory + Story + AN fits in context (max context size - generation size - 20 if generate_until_sentence is enabled)
Right now I'm using api.high_level.generate
where I pass the prompt string. I saw that the method also takes a token array.
So a good approach (in my case, I don't have an Author's Note) would be:
- Tokenize the Memory as
memory[int]
- Tokenize the Story as
story[int]
- Remove items from the start of
story[int]
so thatlen(memory[int])+len(story[int]) <= 8192
if I have the 8192 token limit.
Is that approach correct?
from novelai-api.
So.
-
is about asyncio, you might be doing something wrong somewhere or it could be normal if you get it when the program ends (known asyncio issue).
-
Do and Say are shortcuts to '>You ...' or '>You say "..."'. I don't really remember the exact behavior, but you should see it easily in the context window when using the website (right panel, Advanced, Current Context)
-
Yes, you can have something like an array for Memory, one for Story, and one for AN. Then try to remove the first lines of Story until Memory + Story + AN fits in context (max context size - generation size - 20 if generate_until_sentence is enabled). That's the easiest way of context building.
-
In the website ? The website handles the context building as a whole before sending it to the API (the API knows nothing about the story outside of context), and its complexity is unfathomable. Most of this complexity comes from Lorebook handling, and what to cut, when and how. I tried to replicate this behavior (still WIP), but it's a daunting task and it will never be full-featured.
from novelai-api.
Thanks for answering, I am not getting good responses from text adventure module so I'll probably won't end up using it
from novelai-api.
This issue is stale because it has been open for 30 days with no activity. It will be closed if no activity happens within the next 7 days.
from novelai-api.
I think I can use that approach. I am currently just storing the string in a .txt file and reading from there before using it as prompt.
Do you know the specific place where the web UI inserts the memory string? I want tp place the [Title: ; Tags: ; Genre ;] line that you usually put in memory without hampering with the generation.
Also is there a way to program the responses in a specific structure? For example if I want to generate an item and I want it to generate as "Name: '', class: '', description: ''". Or if I want to generate a specific line like "Health: +20". I want to read these strings and update the database in my game (inventory, stats, etc) accordingly.
from novelai-api.
Do you know the specific place where the web UI inserts the memory string? I want tp place the [Title: ; Tags: ; Genre ;] line that you usually put in memory without hampering with the generation.
Memory is at the very top, AN 3 lines from bottom, Lorebook it depends on insertion order and position.
Also is there a way to program the responses in a specific structure? For example if I want to generate an item and I want it to generate as "Name: '', class: '', description: ''". Or if I want to generate a specific line like "Health: +20". I want to read these strings and update the database in my game (inventory, stats, etc) accordingly.
You could use a "generator", a few-shot example of what you want (there are examples in the official prompts). However, NAI is not very good at math or keeping a state of things like stats. I would recommend doing so externally.
from novelai-api.
This issue is stale because it has been open for 30 days with no activity. It will be closed if no activity happens within the next 7 days.
from novelai-api.
This issue was closed because it has been stale for 7 days with no activity. Reopen it if relevant or open a new issue, further discussion on closed issues might not be seen.
from novelai-api.
Related Issues (20)
- Add support for 3.11
- generate_image return "novelai_api.NovelAIError.NovelAIError: 200 - OK" HOT 8
- Undesired content no longer set properly on image generation HOT 3
- Different results with /ai/annotate-image (controlnet mask generation) HOT 1
- Got the UnicodeDecodeError: 'utf-8' with /ai/generate-image HOT 1
- how to make the generated text always end with a complete sentence? HOT 3
- How to configure loreBook? HOT 2
- Noise for img2img doesn't work as expected
- Presets props are sent to the API, which are not sent by the official UI. HOT 5
- ddim sampler using /ai/generation-image API not working properly on nai-diffusion-v3. HOT 3
- What's the format for inpainting masks? HOT 4
- can not use email and password to login
- Dependency package `sentencepiece (==0.1.99)` dont support `python==3.12` HOT 1
- random tags HOT 7
- Needs ability to decode "document" field in StoryContent HOT 1
- default.preset does not apply. HOT 4
- They have two different base64 HOT 1
- Please add proxy setting to connect api HOT 3
- Could not work well in text Adventure HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from novelai-api.