Error: icdiff: recoverEncode: invalid argument (cannot encode character '\10006')

[lebe-dev]

Error: icdiff: recoverEncode: invalid argument (cannot encode character '\10006')

[lebe-dev]

My env output (sorted): env.txt

[freuk]

This is something to do with utf8 emojis. Instead of reacting to LANG and co, I think it's best to remove them all. I've merged this in #5, so you can run nix profile upgrade packages.x86_64-linux.iter and try again.

[lebe-dev]

I've tried:

sudo nix --extra-experimental-features 'nix-command flakes' profile list
Name:               iter
Flake attribute:    packages.x86_64-linux.iter
Original flake URL: github:freuk/iter
Locked flake URL:   github:freuk/iter/24fb18c78fccd0de8e5c865ca7763ac7fa9bf74e?narHash=sha256-OCsBaf0KcMPrjSf1Ea7N%2B8xZOcObev5Obn4WQFMwUqY%3D
Store paths:        /nix/store/zqvrw2l7vk5xcjdsdr40qkbcv8grbjv0-iter-0.1.0.0

sudo nix --extra-experimental-features 'nix-command flakes' profile upgrade iter

# empty output

Then i've tried to create an app with a prompt from above and got the same error.

[freuk]

Your prompt is fine, I was able to use it.

• Try the nix upgrade again with --refresh to make sure you're getting the latest version (I'm not sure how nix upgrade really behaves yet, as I'm new to it).
• If that doesn't work, It must be that my fix is insufficient - sorry :). Could you try with LANG=C.UTF-8 iter <args> so we can confirm it's an utf8 encoding issue?

[lebe-dev]

Refresh forced to fetch the latest code from git but it didn't helped.

Exported LANG=C.UTF-8 before run iter - won't work.

I'm new to nix too. Asked ChatGPT about utf-8 for nix, it's possible to define encoding at build stage. GPT provided an example:

{ pkgs ? import <nixpkgs> {} }:

pkgs.mkShell {
  buildInputs = [];
  shellHook = ''
    export LANG=en_US.UTF-8
    export LC_ALL=en_US.UTF-8
  '';
}

p.s: my shell (bash+starship) supports emojis

[freuk]

Could you paste your output of running env, so I may be able to construct a reproducer? (edit: don't forget to censor any sensitive parts like.. your GROQ_API_KEY :) )

[freuk]

I tried with this environment and that did not allow me to reproduce the bug at main: neither before #5, nor after it.

\10006 is really the utf8 code for https://www.utf8icons.com/character/10006/heavy-multiplication-x, which is not present in sources at main anymore since #5, so I have some confidence that the fix was right.

Also, your upgrade seems to have worked correctly, because /nix/store/zqvrw2l7vk5xcjdsdr40qkbcv8grbjv0-iter-0.1.0.0 is a store path for a commit after this fix.

I see two possibilities:

• some other version than /nix/store/zqvrw2l7vk5xcjdsdr40qkbcv8grbjv0-iter-0.1.0.0 is being used when you run iter.
• you're infact now hitting a different bug? I recently fixed a "default model name not available" bug in #6.

Could you please update, confirm via nix profile list that your path is now /nix/store/w4j12rz0mwg2qv1ln412226r42ph4xq6-iter-0.1.0.0 (the store path after #6), and then run via /nix/store/w4j12rz0mwg2qv1ln412226r42ph4xq6-iter-0.1.0.0/bin/iter <your arguments> to ensure the right binary is being called?

As an aside, I'm not understanding why your first code snippet starts with a ? and a newline. iter should interpret any newline as an injunction. Or perhaps that's just due to the way the broken encoding represented the utf8 emoji that was supposed to be there.

And.. Thanks for your feedback!

[lebe-dev]

Thank you for patch and investigation. I've implemented your code with python and tried your UI/UX approach. Mission completed :)

[freuk]

Ah well, that's also a way! :) mind sharing the implementation?

…

On Wed, Mar 20, 2024, 11:26 a.m. Eugene Lebedev ***@***.***> wrote: Thank you for patch and investigation. I've implemented your code with python and tried your UI/UX approach. Mission completed :) — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA6URDGXF27RMDNFASTTZ7LYZG2DFAVCNFSM6AAAAABEU4GYW6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBZHE4DOOJZGE> . You are receiving this because you commented.Message ID: ***@***.***>

[lebe-dev]

The first goal with your project was to try your UI/UX approach. AI in every corner but I don't known even one approach to apply it to existing code base (cody, copilot, phythogora, etc. are not convenient and helpful yet).

I like your approach to show two panes with before and after. But imo it's not suitable for projects with multiple files with cross dependencies between multiple files. I've tried to implement the same, tested it and forget. Don't want to make research in such direction right now, my code is too raw to publish it.

Anyway thanks for the idea of UI/UX in iter tool.

My current approach to work with existing projects - use bash script which reads all project files (exclude some files and directories) and generates huge prompt for Claude 3 Opus (work with any LLM). Then i work with project, add modifications.

[freuk]

Yeah, I agree this is still too limiting. I think we probably want to have a more "dashboard-ish" UX to leverage the LLM more. One idea is to ask that the LLM return an additional json where it self-rates its answer according to several criteria, and these criteria can be visualized directly on a graph, for example. Text is still too slow to read :) Cheers.

…

On Sat, Mar 23, 2024, 10:21 a.m. Eugene Lebedev ***@***.***> wrote: The first goal with your project was to try your UI/UX approach. AI in every corner but I don't known even one approach to apply it to existing code base (cody, copilot, phythogora, etc. are not convenient and helpful yet). I like your approach to show two panes with before and after. But imo it's not suitable for projects with multiple files with cross dependencies between multiple files. I've tried to implement the same, tested it and forget. Don't want to make research in such direction right now, my code is too raw to publish it. Anyway thanks for the idea of UI/UX in iter tool. My current approach to work with existing projects - use bash script which reads all project files (exclude some files and directories) and generates huge prompt for Claude 3 Opus (work with any LLM). Then i work with project, add modifications. — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA6URDBXUG6KM5SECVNYFO3YZWMY5AVCNFSM6AAAAABEU4GYW6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJWGUZDGNBWGI> . You are receiving this because you commented.Message ID: ***@***.***>