kaitai-io / kaitai_struct_webide Goto Github PK
View Code? Open in Web Editor NEWOnline editor / visualizer for Kaitai Struct .ksy files
Home Page: https://ide.kaitai.io
License: GNU General Public License v3.0
Online editor / visualizer for Kaitai Struct .ksy files
Home Page: https://ide.kaitai.io
License: GNU General Public License v3.0
This way the WebIDE could save / load .ksy and input binary files directly from filesystem, so we don't have to upload / download files manually.
(If the local folder is synced then you can get version history, push directly to Git, etc.)
Make it possible and document an easy way to stop on exceptions (but skip compiler exceptions).
I don't know exact details, but probably parsing a complex file can use all the allowed stack up in Chrome.
(@xavrr mentioned the issue on Gitter: "is there a way to increase the stack size for chrome ? to allow for bigger size parsing ?")
Currently, hex viewer always shows 16 bytes (0x10 bytes) per line. On bigger screens, it might be more comfortable to dump more bytes per line (i.e. 20, 24, 32 or even more). Can we try to make it a configurable setting?
TODO:
doc
contents when navigating the tree (in status bar / description field / something like that)Not planned in the near future:
Not planned (until explicit user request):
Done:
Is there any open-source License for this project? There should be a LICENSE file in the root.
Opening https://ide.kaitai.io/ in Firefox Nightly on Windows 7 64 bit, selecting formats/archive/cpio_old_le.ksy
and I see this error and empty object tree
Parse error: undefined
Call stack: undefined io.kaitai.struct.format.YAMLParseException: /types/file/seq/6/if: can't compare BytesLimitType(BinOp(Attribute(Name(identifier(header)),identifier(path_name_size)),Sub,IntNum(1)),None,false,None,None) and CalcBytesType
Make the WebIDE able to handle cases where custom processing is used like in this example: kaitai-io/kaitai_struct#137 (comment)
Current color scheme of web IDE is suboptimal: some of the widgets are clearly in "dark" theme (i.e. bright foreground on dark background), and some of them are in "bright" theme (i.e. the opposite). It's kind of hard for the eyes, as you can't get used to either of them.
I want to suggest that we could:
require.js times out after ~30 seconds. Tries to reload every resource on refresh and times out again and again => no trivial workaround is known to solve this.
It turns out that there are many more timestamp formats in the wild besides UNIX timestamp. It would be cool to show more of them in "converter" to aid timestamp detection guesswork.
Unix timestamp
Microsoft FILETIME
WebKit/Chrome timestamp
Apple Cocoa Core Data timestamp
Apple Mac OS X HFS+ timestamp
Seconds since year 0
Excel timestamp
Microsoft FAT date time
Microsoft SYSTEMTIME
ISO9660 decimal timestamp
ISO9660 binary timestamp
The converter panel counts bits (u32le) whereas KSY uses bytes (u4le). It also has separate columns for signed and unsigned values but labels rows with the signed type only.
When reversing a format you often look at hex, trying to guess size based on visual perception, and look for that size (trying any endianness and any integer size in worst case) trying to locate the field which it stores. But there is no such a feature in webide.
If we have 16 bytes that are supposed to be GUID/UUID, I propose to add some hint that would allow us to display it in its canonical text form, i.e. stuff like 123e4567-e89b-12d3-a456-426655440000
.
Currently we need hacks like that to get some sort of approximation:
types:
uuid:
seq:
- id: p1
type: u4
- id: p2
type: u2
- id: p3
type: u2
- id: p4
type: u2be
- id: p5
size: 6
-webide-representation: "{p1}-{p2}-{p3}-{p4}-{p5}"
However, it's better if we could get rid of special UUID type just for display purposes and get WebIDE to represent any 16-byte raw byte array as UUID using something like
-webide-representation: "{some_attribute:uuid}"
or something like that.
Some formats depend on other .ksy format files.
The IDE should be modified in a way that we can load multiple ksy files for parsing even if we only use a single .ksy file for editing.
One of the possible approaches is to create a "loaded classes" window where we can control which classes are auto-loaded before the parsing happens. Here we should be able load, unload single classes and unload every currently loaded classes too.
It is damn slow when I enter a char and it tries to compile an invalid ksy and parse a large file with it. It's better to compile and parse by ctrl+enter.
Loading the RAR format and parsing a RAR5 file (even with an empty file, like this sample) will crash the Chrome tab renderer, because eating too much memory probably.
I'm looking at the demo site install at https://ide.kaitai.io/ and I attempted to upload a JPEG file from disk. The errors pane reports:
KS compilation error: io.kaitai.struct.format.YAMLParseException: /types/exif_in_jpeg/seq/1: unable to find type 'exif', searching from jpeg
The JPEG is well formed and has exif data.
We need feature usage metrics to make wiser development decisions.
We'll probably use Google Analytics for this purpose and it would be good if we could warn the user about this, explain why we use it and give opt-out option (other than blocking it by browser extension).
Try https://github.com/kaitai-io/kaitai_struct_formats/blob/master/scientific/nt_mdt/nt_mdt.ksy with http://www.ntmdt-si.ru/data/media/files/scangallery/graphene_and_carbon_materials/graphene_2.mdt
On 2 GiB RAM system it causes browser and GUI going unresponsive because of swapping.
Even if it is the result of a loop, I guess we need something to prevent this. For example webide can have some fields to limit amount of ram and stop parsing when the limit is hit.
P.S. python-compiled version works OK on that file.
Load and save files (commit?) directly from a Github repo.
js runtime relies on nodejs zlib
+require
module, neither are available in the browser.
Proposed fix:
Distribute the kaitai_struct_webide
as Electron application and npm module. it should allow us install the kaitai_struct_ide as npm package, something like that: npm install kaitai
(alternative names: npm install kaitai_struct
or npm install kaitai_struct_ide
).
Here is an example of wrapping it with electron:
#!/usr/bin/env bash
set -euo pipefail
git clone --depth 1 https://github.com/kaitai-io/kaitai_struct_webide.git
cd kaitai_struct_webide
cat <<'OUT' > package.json
{
"name": "kaitai_struct_webide",
"description": "Web IDE for Kaitai Struct",
"directories": {
"doc": "docs"
},
"dependencies": {
"typescript": "^2.2.1",
"electron": "latest"
},
"main": "main.js",
"devDependencies": {},
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"start-server": "python2 serve.py &",
"start": "npm run-script start-server && node_modules/.bin/electron ."
},
"repository": {
"type": "git",
"url": "git+https://github.com/kaitai-io/kaitai_struct_webide.git"
},
"author": "",
"license": "GPL-3.0",
"bugs": {
"url": "https://github.com/kaitai-io/kaitai_struct_webide/issues"
},
"homepage": "https://github.com/kaitai-io/kaitai_struct_webide#readme"
}
OUT
npm install
Create slightly changed kaitai_struct_webide/main.js from electron quickstart:
const electron = require('electron')
// Module to control application life.
const app = electron.app
// Module to create native browser window.
const BrowserWindow = electron.BrowserWindow
const path = require('path')
const url = require('url')
// Keep a global reference of the window object, if you don't, the window will
// be closed automatically when the JavaScript object is garbage collected.
let mainWindow
function createWindow () {
// Create the browser window.
mainWindow = new BrowserWindow({width: 800, height: 600})
// and load the index.html of the app.
mainWindow.loadURL('http://localhost:8000');
// Open the DevTools.
// mainWindow.webContents.openDevTools()
// Emitted when the window is closed.
mainWindow.on('closed', function () {
// Dereference the window object, usually you would store windows
// in an array if your app supports multi windows, this is the time
// when you should delete the corresponding element.
mainWindow = null
})
}
// This method will be called when Electron has finished
// initialization and is ready to create browser windows.
// Some APIs can only be used after this event occurs.
app.on('ready', createWindow)
// Quit when all windows are closed.
app.on('window-all-closed', function () {
// On OS X it is common for applications and their menu bar
// to stay active until the user quits explicitly with Cmd + Q
if (process.platform !== 'darwin') {
app.quit()
}
})
app.on('activate', function () {
// On OS X it's common to re-create a window in the app when the
// dock icon is clicked and there are no other windows open.
if (mainWindow === null) {
createWindow()
}
})
// In this file you can include the rest of your app's specific main process
// code. You can also put them in separate files and require them here.
Then run with npm start
.
It should work but it shows the Your browser is not supported. Features may or may not working. Please use the latest Chrome or Firefox
message. It is shown in other browsers too (e.g. Firefox 52), so I assume we need to check compatibility.
Setup:
Windows 64 bit,
Python 2.7.13 |Anaconda 4.4.0 (64-bit)| (default, May 11 2017, 13:17:26) [MSC v.
1500 64 bit (AMD64)] on win32
node: v6.11.3
tried typescript: 2.5.2
I've followed the steps to compile web ide locally on Windows 7 but failed due to this error.
C:\Users\test\kaitai_struct_webide>python serve.py --compile auto compile started please use 127.0.0.1:8888 on Windows (using localhost makes 1sec delay) Changed file: ./src\worker\TemplateCompiler.ts 'tsc' is not recognized as an internal or external command, operable program or batch file. Compile errors: -
C:\Users\test\kaitai_struct_webide>npm install -g typescript
C:\Users\test\kaitai_struct_webide>python serve.py --compile auto compile started Traceback (most recent call last): File "serve.py", line 111, in <module> import genKaitaiFsFiles File "C:\Users\iscamc\kaitai_struct_webide\genKaitaiFsFiles.py", line 21, in < module> with open(outDir + 'js/kaitaiFsFiles.js', 'wt') as f: IOError: [Errno 2] No such file or directory: '--compile/js/kaitaiFsFiles.js'
outDir = sys.argv[1] + '/' if len(sys.argv) > 1 else ''
outDir = sys.argv[1] + '/' if len(sys.argv) > 2 else ''
Any thoughts how to resolve this?
About box has a link to https://ide.kaitai.io/LICENSE-3RD-PARTY.txt — which is 404 now. Probably something's not built and/or deployed properly in our CI deployment process.
Because it is actually stored in the file system with the original name... And the FsItem should be used instead of the fn...
As a workaround first at least do not auto-parse at startup if the previous parse crashed the browser (ie. the previous parse started but never finished).
Hi!
I'm loving the web IDE so far, but it seems imports aren't working.
If I copy the imports example from http://doc.kaitai.io/user_guide.html#_importing_types_from_other_files and open the filelist.ksy
file in the IDE, the object tree is not updated (it stays blank or open on the previous ksy file) but there are no errors visible in the IDE.
If I open the browser console, I can see these errors:
As text:
localforage.js:836 [performance/29.891] YAML parsing took 1 milliseconds.
localforage.js:836 import yaml: date, mode: rel
localforage.js:836 import yaml: date, mode: rel
localforage.js:836 GET https://ide.kaitai.io/formats/date.ksy 404 ()
send @ jquery-3.1.1.min.js:4
ajax @ jquery-3.1.1.min.js:4
get @ /js/app.files.js:55
(anonymous) @ /js/app.js:37
importYaml @ /js/app.js:35
$c_Lio_kaitai_struct_format_JavaScriptClassSpecs.doImport__T__T__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:68200
$c_Lio_kaitai_struct_precompile_LoadImports.io$kaitai$struct$precompile$LoadImports$$loadImport__T__sci_List__s_Option__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:7953
(anonymous) @ kaitai-struct-compiler-fastopt.js:7825
$c_Lio_kaitai_struct_precompile_LoadImports.processClass__Lio_kaitai_struct_format_ClassSpec__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:7838
$c_Lio_kaitai_struct_Main$.importAndPrecompile__Lio_kaitai_struct_format_ClassSpecs__Lio_kaitai_struct_RuntimeConfig__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:4719
$c_Lio_kaitai_struct_MainJs$.compile__T__sjs_js_Object__Lio_kaitai_struct_JavaScriptImporter__Z__sjs_js_Promise @ kaitai-struct-compiler-fastopt.js:4892
$c_Lio_kaitai_struct_MainJs$.$$js$exported$meth$compile__T__sjs_js_Object__Lio_kaitai_struct_JavaScriptImporter__Z__O @ kaitai-struct-compiler-fastopt.js:4821
$c_Lio_kaitai_struct_MainJs$.compile @ kaitai-struct-compiler-fastopt.js:4945
compile @ /js/app.js:100
copyPromise.then.then @ /js/app.js:127
Promise resolved (async)
copyPromise.then @ /js/app.js:126
Promise resolved (async)
localforage.getItem.then.ksyFsItem @ /js/app.js:123
Promise resolved (async)
req.onsuccess @ localforage.js:836
IndexedDB (async)
(anonymous) @ localforage.js:826
localforage.js:836 XHR finished loading: GET "https://ide.kaitai.io/formats/date.ksy".
send @ jquery-3.1.1.min.js:4
ajax @ jquery-3.1.1.min.js:4
get @ /js/app.files.js:55
(anonymous) @ /js/app.js:37
importYaml @ /js/app.js:35
$c_Lio_kaitai_struct_format_JavaScriptClassSpecs.doImport__T__T__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:68200
$c_Lio_kaitai_struct_precompile_LoadImports.io$kaitai$struct$precompile$LoadImports$$loadImport__T__sci_List__s_Option__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:7953
(anonymous) @ kaitai-struct-compiler-fastopt.js:7825
$c_Lio_kaitai_struct_precompile_LoadImports.processClass__Lio_kaitai_struct_format_ClassSpec__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:7838
$c_Lio_kaitai_struct_Main$.importAndPrecompile__Lio_kaitai_struct_format_ClassSpecs__Lio_kaitai_struct_RuntimeConfig__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:4719
$c_Lio_kaitai_struct_MainJs$.compile__T__sjs_js_Object__Lio_kaitai_struct_JavaScriptImporter__Z__sjs_js_Promise @ kaitai-struct-compiler-fastopt.js:4892
$c_Lio_kaitai_struct_MainJs$.$$js$exported$meth$compile__T__sjs_js_Object__Lio_kaitai_struct_JavaScriptImporter__Z__O @ kaitai-struct-compiler-fastopt.js:4821
$c_Lio_kaitai_struct_MainJs$.compile @ kaitai-struct-compiler-fastopt.js:4945
compile @ /js/app.js:100
copyPromise.then.then @ /js/app.js:127
Promise resolved (async)
copyPromise.then @ /js/app.js:126
Promise resolved (async)
localforage.getItem.then.ksyFsItem @ /js/app.js:123
Promise resolved (async)
req.onsuccess @ localforage.js:836
IndexedDB (async)
(anonymous) @ localforage.js:826
/#:1 Uncaught (in promise) Object {readyState: 4, getResponseHeader: function, getAllResponseHeaders: function, setRequestHeader: function, overrideMimeType: function…}
Promise rejected (async)
j @ jquery-3.1.1.min.js:2
k @ jquery-3.1.1.min.js:2
setTimeout (async)
(anonymous) @ jquery-3.1.1.min.js:2
i @ jquery-3.1.1.min.js:2
fireWith @ jquery-3.1.1.min.js:2
fire @ jquery-3.1.1.min.js:2
i @ jquery-3.1.1.min.js:2
fireWith @ jquery-3.1.1.min.js:2
A @ jquery-3.1.1.min.js:4
(anonymous) @ jquery-3.1.1.min.js:4
XMLHttpRequest.send (async)
send @ jquery-3.1.1.min.js:4
ajax @ jquery-3.1.1.min.js:4
get @ /js/app.files.js:55
(anonymous) @ /js/app.js:37
importYaml @ /js/app.js:35
$c_Lio_kaitai_struct_format_JavaScriptClassSpecs.doImport__T__T__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:68200
$c_Lio_kaitai_struct_precompile_LoadImports.io$kaitai$struct$precompile$LoadImports$$loadImport__T__sci_List__s_Option__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:7953
(anonymous) @ kaitai-struct-compiler-fastopt.js:7825
$c_Lio_kaitai_struct_precompile_LoadImports.processClass__Lio_kaitai_struct_format_ClassSpec__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:7838
$c_Lio_kaitai_struct_Main$.importAndPrecompile__Lio_kaitai_struct_format_ClassSpecs__Lio_kaitai_struct_RuntimeConfig__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:4719
$c_Lio_kaitai_struct_MainJs$.compile__T__sjs_js_Object__Lio_kaitai_struct_JavaScriptImporter__Z__sjs_js_Promise @ kaitai-struct-compiler-fastopt.js:4892
$c_Lio_kaitai_struct_MainJs$.$$js$exported$meth$compile__T__sjs_js_Object__Lio_kaitai_struct_JavaScriptImporter__Z__O @ kaitai-struct-compiler-fastopt.js:4821
$c_Lio_kaitai_struct_MainJs$.compile @ kaitai-struct-compiler-fastopt.js:4945
compile @ /js/app.js:100
copyPromise.then.then @ /js/app.js:127
Promise resolved (async)
copyPromise.then @ /js/app.js:126
Promise resolved (async)
localforage.getItem.then.ksyFsItem @ /js/app.js:123
Promise resolved (async)
req.onsuccess @ localforage.js:836
IndexedDB (async)
(anonymous) @ localforage.js:826
localforage.js:836 GET https://ide.kaitai.io/formats/date.ksy 404 ()
send @ jquery-3.1.1.min.js:4
ajax @ jquery-3.1.1.min.js:4
get @ /js/app.files.js:55
(anonymous) @ /js/app.js:37
importYaml @ /js/app.js:35
$c_Lio_kaitai_struct_format_JavaScriptClassSpecs.doImport__T__T__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:68200
$c_Lio_kaitai_struct_precompile_LoadImports.io$kaitai$struct$precompile$LoadImports$$loadImport__T__sci_List__s_Option__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:7953
(anonymous) @ kaitai-struct-compiler-fastopt.js:7825
$c_Lio_kaitai_struct_precompile_LoadImports.processClass__Lio_kaitai_struct_format_ClassSpec__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:7838
$c_Lio_kaitai_struct_Main$.importAndPrecompile__Lio_kaitai_struct_format_ClassSpecs__Lio_kaitai_struct_RuntimeConfig__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:4719
$c_Lio_kaitai_struct_MainJs$.compile__T__sjs_js_Object__Lio_kaitai_struct_JavaScriptImporter__Z__sjs_js_Promise @ kaitai-struct-compiler-fastopt.js:4892
$c_Lio_kaitai_struct_MainJs$.$$js$exported$meth$compile__T__sjs_js_Object__Lio_kaitai_struct_JavaScriptImporter__Z__O @ kaitai-struct-compiler-fastopt.js:4821
$c_Lio_kaitai_struct_MainJs$.compile @ kaitai-struct-compiler-fastopt.js:4945
compile @ /js/app.js:101
copyPromise.then.then @ /js/app.js:127
Promise resolved (async)
copyPromise.then @ /js/app.js:126
Promise resolved (async)
localforage.getItem.then.ksyFsItem @ /js/app.js:123
Promise resolved (async)
req.onsuccess @ localforage.js:836
IndexedDB (async)
(anonymous) @ localforage.js:826
localforage.js:836 XHR finished loading: GET "https://ide.kaitai.io/formats/date.ksy".
send @ jquery-3.1.1.min.js:4
ajax @ jquery-3.1.1.min.js:4
get @ /js/app.files.js:55
(anonymous) @ /js/app.js:37
importYaml @ /js/app.js:35
$c_Lio_kaitai_struct_format_JavaScriptClassSpecs.doImport__T__T__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:68200
$c_Lio_kaitai_struct_precompile_LoadImports.io$kaitai$struct$precompile$LoadImports$$loadImport__T__sci_List__s_Option__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:7953
(anonymous) @ kaitai-struct-compiler-fastopt.js:7825
$c_Lio_kaitai_struct_precompile_LoadImports.processClass__Lio_kaitai_struct_format_ClassSpec__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:7838
$c_Lio_kaitai_struct_Main$.importAndPrecompile__Lio_kaitai_struct_format_ClassSpecs__Lio_kaitai_struct_RuntimeConfig__s_concurrent_Future @ kaitai-struct-compiler-fastopt.js:4719
$c_Lio_kaitai_struct_MainJs$.compile__T__sjs_js_Object__Lio_kaitai_struct_JavaScriptImporter__Z__sjs_js_Promise @ kaitai-struct-compiler-fastopt.js:4892
$c_Lio_kaitai_struct_MainJs$.$$js$exported$meth$compile__T__sjs_js_Object__Lio_kaitai_struct_JavaScriptImporter__Z__O @ kaitai-struct-compiler-fastopt.js:4821
$c_Lio_kaitai_struct_MainJs$.compile @ kaitai-struct-compiler-fastopt.js:4945
compile @ /js/app.js:101
copyPromise.then.then @ /js/app.js:127
Promise resolved (async)
copyPromise.then @ /js/app.js:126
Promise resolved (async)
localforage.getItem.then.ksyFsItem @ /js/app.js:123
Promise resolved (async)
req.onsuccess @ localforage.js:836
IndexedDB (async)
(anonymous) @ localforage.js:826
/#:1 Uncaught (in promise) Object {readyState: 4, getResponseHeader: function, getAllResponseHeaders: function, setRequestHeader: function, overrideMimeType: function…}
It looks like it's trying to retrieve the date.ksy
file from https://ide.kaitai.io/formats/date.ksy
, but that's returning a 404 which is not caught.
Kaitai WebIDE version: 0.1.0.2207 (license)
Kaitai compiler version: 0.7 (2017-04-26 20:06:21.086)
Chrome version 58.0.3029.110 (64-bit)
Windows 10 64-bit
A few people already mentioned that: why can't we work with something tree-like in Web IDE, instead of just a text editor with a YAML file?
Instead of editing (and actually re-parsing YAML on every occassion), we could provide tree-like UI and have some basic operations, like "Add / Edit Attribute", "Add Type", "Rename / Move Type", etc, with some wizard-like pop-up windows. Tehcnically, YAML is already a tree, and we already have a tree UI library used for tree representation of the data, so why don't we reuse it?
I wanted to install webide on the android tablet using termux (I have tested webide remote version and find it quite useful, it looked to me to be less complicated to install with termux than cmd line version, as termux have no jde available). However I run into problem with installation and launching python scripts, which are written for python2, and only python3 comes with termux. Python3 is becoming a standard version in the many linux distributions and many software is dropping python 2 support.
So I can offer my time to modify python code and add PR, the main question is how? These are to considerations:
py2to3
.six
so that it would work with both python2 and python3.PS. I am really impressed with capabilities of kaitai_struct (even if used only as the tool to RE binary format) it is manyfold better tool compared with other hex editors. I have some highly dynamic binary formats in my RE experience (some scientific electron microscopy formats, including virtual file system) and using Hexinator or wxHexEditor were really painful on dynamic data structs. Lately I am RE another binary format and find that using traditional hexeditors I am not able to manage whole complexity, and started to translate my knowledge about format into ksy
. I wish kaitai_struct were available for me two years ago, I would had not lost so much time. I am still not sure about compiled code, as excessive class construction and hierarchical instance access is costly in python. Soon I will have something to compare.
Create built-in ksy syntax analyzer based on the Ksy Style Guide (kaitai-io/kaitai_struct#140).
I'd like to propose a "watch" functionality. It should probably work like that:
foo.bar.someArray[42].someOtherElement
- and user can edit itBonus:
7
is encountered 15 times, result 1
is encountered 4 times, result 5
if encountered twice, etc).Make compiler version configurable.
Stable should default to the latest stable compiler version, while devel should default the latest snapshot compiler version.
If the loaded .ksy's version is higher than the loaded compiler's version then suggest to the user to switch compiler versions.
Currently when opaque types/processes are used the script fails because it does not have definitions for opaque items. It would be nice if .js files could be loaded into the Web IDE and used to "fill in" opaque types and processes during KSY development. The Web IDE could scan the JS Code (debug) for opaque items by looking for "require('./SOME_FILE.js')" to extract the file name then identify if this external item has a KSY file. If it does then do nothing. If it does not then scan for a corresponding .js file and if found then copy js file contents to bottom of JS Code (Debug). The re-parsing would then work and allow users to test opaque items.
https://gist.github.com/DarkDimius/9c8e32c9912d294600aa1d875cdb52d1 - ksy
https://yadi.sk/d/JkoqyX2A3ENycN - binary
https://yadi.sk/i/DZC3t9yc3EP6Xg - screenshot
quoting @koczkatamas
yeah, it's a bug (or a missing feature), what happens here is that the 'decoded' property is an instance, so I don't know the type of it only after processing (expanding) it, but after doing that I don't modify the node anymore, only add child nodes
Although it looks like the new file is selected, when you modify the new file actually the old file is modified.
Candidate lib: https://github.com/APIs-guru/graphql-voyager
Hi all, I'm not sure it's an issue or expected behaviour but the fact is that I didn't have this issue in the web ide or vsc so I wanted to share with you:
I have this kind os sequence in my ksy:
- id : value1
type : u1
if : header.value1bit == true
- id : value2
type : u4
if : value1 == 0
when evaluating value1 == 0 the csharp code wont check that it exists in the curent parsing context (only if header.value1bit == true) and would then evaluate value1==0 as true and the parsing is crashing.
I had to change it to:
- id : value2
type : u4
if : header.value1bit == true and value1 == 0
I have the feeling that as we are referencing "value1" in the second expression, it should implicitly check that "value1" exists, kind of inherit that from the first expression...
Currently we use a non-standard font and the WebIDE looks slightly different on non-Windows based systems.
As there is no guarantee that these fields will be supported in the future.
This is valid YAML/KSY that contains a flow-folded string[1][2][3].
meta:
id: example
file-extension: example
seq:
- id: stringdemo
size: 0
doc: This is a flow-style
multi-line string
But causes the web IDE's YAML parser to throw a ParseException
:
YAML parsing error: <ParseException> Unable to parse. (line 8: ' multi-line string')
Quoting the string (with '
or "
) causes the parser to throw this exception instead:
YAML parsing error: <ParseException> Malformed inline YAML string ('This is a flow-style). (line 7: 'doc: 'This is a flow-style ')
Add option to parse
as a defined type separately.
This can help quick prototyping and limit the possible external errors while parsing a complex file format.
When reverse-engineering a format you often need to put a structure on some offset. That offset can be determined by other members offsets and sizes. So I propose to add a feature with the following look&feel:
1 a user selects the size or pos of the field to be modified and the depth to which changes will be propagated. Webide goes up n levels of structs (0 - the struct where the chosen field in, 1 the structure containing the structure where the field is in), cuts the piece of binary, cuts the needex pieces of ksy, moves it into a separate tab. This will allow to experiment solely on that piece without screwing the rest of parsed binary.
2 user moves the structures after (in the cace of size) or in (in the case of pos) that field with mouse and/or keyboard. Moving is counted as offset to the initial position determined by ksy file. When user feels that the position is correct, he stops, WebIde modifies ksy.
3 then modified ksy can be merged back to the ksy it was ripped from.
Also it may be useful to show the bytes not as blocks with hex as colorful blocks , and structures as colorful frames around this blocks. This may allow easier spotting of patterns in data.
The project uses a lot of 3rd party libraries now. Probably we'll need some clear "licensing" text, that will contain all of these licenses with references to particular files, copyright holders, etc, etc.
When one's working on a moderately large project, it is often impractical to view the whole tree of resulting objects, when you're working on something deep inside it.
It would be cool if tree view window would allow to choose expression to visualize. I.e. by default it is something like obj
(and it recursively allows to travel through whole tree), but I can choose it show the tree starting from
obj.foo.bar.baz[3]
or, even more complex example, something like
obj.records.filter(r => r.header.type == 42).map(r => r.body.data)
It should be relatively easy to do, if we'll just traverse object resulting from this expression, and not default object?
Maybe add iconv-lite support too?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.