pouyakary/orchestra

One language to be RegExp's Successor. Visually readable and rich, technically safe and extended, naturally scalable, advanced, and optimized

Orchestra

Orchestra is a visual language that compiles into RegExp (as for ECMAScript.). It's replacement for the existing syntax of Regular Expressions and tries to build upon that a better language so developers can do more powerful stuff while having less pain.

We also made Orchestra for ourselves in the first place which means we wanted it to be the best tool possible, therefore we added every feature that we thought is helpful and every tool we used in our production. We don't want people to be forced to do something in Orchestra and then configure the rest of their tools to work with the result, Orchestra is designed so that you do something and use it in production, nothing in between. All the popular tools for RegExp like RegExr, RegExpU, Babel and Regulex are integrated into Orchestra or we have made Orchestra in a way that you no longer need them. So all you have to do is to make the edit and use the compiled regexp in production.

Features

Orchestra Language

It's a truly new language that gives you all sorts of possibilities and features you want from a powerful regular expression system.

• Readability
  The wildcard syntax of Regular Expression engines are very problematic when it comes to their readability. Orchestra solves this problem by introducing a new syntax that is visual, very readable and maintainable while still is fast to write.

• Scalability
  Orchestra is capable of writing very large regular expressions. With that, you can write scalable and maintainable regular expressions suited for doing a large task. You can now use RegExps for a task like data mining, parser writing and so on...

• Safe & Optimized Compilation
  Orchestra compiles to optimized and safe RegExp. It takes care of escapes and Unicode text encodings. You can have things like Emojies in your phrases and it'll work! Orchestra's Compiler also embraces RegExpU (the RegExp to ES5 transformer used inside of Babel) so you can be sure that what you compile will work everywhere.

• Higher Level Language Coming in Version 2
  Within the second release you can expect to have variables, functions, modules, and libraries in Orchestra. It will truly change the face of regular expressions by giving you the tools to write really powerful RegExes. With that, you should be expecting many new applications for regular expressions to be used in.

Orchestra Studio

Orchestra Studio is the IDE for Orchestra Language. As Orchestra is a visual language it's IDE and Language Core are very much together and it's very hard for Orchestra Language to exists outside of the Orchestra Studio. Because of that, we tried to make Orchestra Studio the best IDE for developing regular expressions possible. It features:

• Great Editor for Orchestra
  There really can't be any better IDE for Orchestra possible. It compiles Orchestra codes as you change to edit and when you select an Orchestra block it highlights the matching part in the compiled RegExps so that you can see what you change are.

• Importing RegExps
  By using Kary Foundation's Concerto Compiler, Orchestra Studio is capable of importing any RegExp code and convert it to an optimized and greatly formatted Orchestra code. So you can import and edit any of your existing RegExps.

• Great Debugging with Playgrounds
  Orchestra Studio lets you define a sample "testing text" and many quick tests in your Orchestra project and each time you change your Orchestra code you can instantly view how it executed on your samples and what it did find there. We designed playground with all the benefits you might have in RegExr (match highlights, hover on match to get information) and what's even better is that your playground texts also get saved within the Orchestra file, therefore, each time you open your file your tests get loaded and you can check your work as you make changes.

• Test Cases Coming in Version 2
  With this feature you'll be able to define tests to say if a certain text must be matched or must be rejected and also what the match groups must contain and hence each time you make changes you'll see if your changes have broken any tests... (by each single change...)

• Best Documentations
  Orchestra comes with an offline documentation for Orchestra Language and is integrated right into the Orchestra Language. You can right click on any block and select 'help' to view the block's full documentation.

• RegExp Visualizations Coming in Version 2
  Orchestra Studio visualizes your resulting RegExp sequence and lets you walk through the matching phases using Regulex. If you use Unsafe Wildcard feature in your Orchestra's that is going to be the best way to see how you have changed your code.

• Great IDE
  Orchestra is a beautiful IDE. It contains the perfection that Kary Foundation software share. Beautiful UI and easy UX makes it easy-to-use and enjoyable to spend-time-in. It has night/light modes so you can work at any time in the day. It also is cross-platform and runs on macOS, Linux and Window systems.

Big Disclaimer

Currently, we are working on a prototype version of Orchestra trying to find out the best shape for the software so some parts of the source code are not as pretty as they should be and our commits aren't the best commits one can make. Also, there might be buttons with no functionality or empty parts which you can expect from a work in the heavy development phase. For software in this phase, people mostly commit in private and make it public once it's ready. We believe in transparent development so we make this phase publicly visible but you have to be aware of what is the status of the repo. As we hit the first beta release this all will be over and you'll have a stable repo.

orchestra's Issues

Concerto: Lookahead is broken

Because of the way Regulex designed it's parser we have to have a system to look for the next node and if lookahead perform both nodes together, not like this:

Invalid filename on windows: docs\How To Build?.md

How To Build?.md has a question mark in the filename, which prevents the file being checked out from the repo. Windows does not allow filenames to contain question marks.

Wrong escaping of \/

/^(?:[0-9]){4}\\/(?:[0-9]){2}\\/(?:[0-9]){2}/

npm install fails on Debian

When I run npm install I get the following error because of macos-alias package:

npm ERR! code EBADPLATFORM
npm ERR! notsup Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm ERR! notsup Valid OS:    darwin
npm ERR! notsup Valid Arch:  any
npm ERR! notsup Actual OS:   linux
npm ERR! notsup Actual Arch: x64

Concerto: Row Text needs optimization

RegExp Unicode Property Exscapes (Proposals from TC39's upcoming draft and UTS18 RL1.2) to be implemented in QuartetCore

Introduction

RegExp Unicode Property Escapes are new parts of JavaScript RegExp tools we need to have in order to support the upcoming JavaScript editions. With this coming then we have to have two different generators for the plain text inputs in the QuartetCore that results having a transpiler like Babel and TypeScript to convert future editions of JS (ES7 or ES8 maybe) into ES5/ES6 editions in order to keep our backward-compatibility support.

Also this means that the architecture of the blocks must be configurable so that in run-time we can add or remove blocks from the toolbox for different language supports

Supporting other languages

We also have to have full compatibility with UTS18 RL1.2 in order to support other languages

Block Changes

Static blocks

The other thing to have for this big change is to have for example the Digit block be compiled to different things with or without Unicode support:

// in case of normal compile:
/\d/

// in case of Unicode in mind compile:
/\p{Decimal_Number}/u

Now thin actually means a lot. Because maybe the use is not after having the all Unicode chars matched within the context and only basic Latin ones (/[0-9]/) to be matched. This is the point where the blocks must be redesigned.

Compiler Implementation

The best option is to compile to fully ES6 RegExps and then have the code to be down-compiled to ES5 if needed with tools like RegExpU

JS regex archive

Here's a collection that you may find helpful:
https://github.com/regexhq

Based on #19 there is a need for back reference block

Back Reference block was not added to Orchestra (and causes problems: #19) because there needed to be a good design for it. Back-reference can be as simple as:

The problem here is: there is no guarantee of safety of the number. So one might suggest using options to remove the possibility of unsafe numbers:

However this is also a very bad practice as one can not determine the group numbers within a range:

Another problem here is the name groups, in the future we can support named captures as:
(<name>something) and this can also be viewed as:

So clearly there is a need for a good idea towards implementing this

window.scrollTo / window.scrollBy not working

That's it, it's not working, so both window.scrollTo and jump functions are broken and help page can't scroll

Chunk Reduce Optimizing algorithm has bugs

This is what reduces:
(?:(?:<...>)) to (?:<...>)

But it has this problem:

Workspace view resize problem

There was a bug within the views that if you were in the playground view and resized the screen, once you got back to editor you would see no editor. That happened was that when switch happens between tabs other views get to be hidden thus the resize would have no effect on Blockly. That's why the resize had to be done manually at the time of the switch:

    function onChangeWindowToEditor ( ) {
        CurrentActiveView = 'editor';
        document.getElementById( EditorWindowScreenElement ).hidden = false;
        document.getElementById( PlaygroundWindowScreenElement ).hidden = true;
        document.getElementById( TabsPlaceholder ).className = 'editor-tab';

        Blockly.fireUiEvent( window, 'resize' );
    }

FIX COMMIT

Unclear installation instructions

it's not clear how to build / run orchestra IDE
what is system dependencies and how to install them ?
how to build orchestra IDE itself ?

PS : Great job btw!

Doesn't run on Windows

During install:

npm install
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as possible. Use 'npm ls graceful-fs' to find it in the tree.

npm run build
[15:43:30] Starting 'sheets'...
{ Error: Command failed: git rev-list --all --count > _compiled/about/commit-count.txt
The system cannot find the path specified.

    at ChildProcess.exithandler (child_process.js:198:12)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at maybeClose (internal/child_process.js:891:16)
    at Socket.<anonymous> (internal/child_process.js:342:11)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at Pipe._handle.close [as _onclose] (net.js:497:12)
  killed: false,
  code: 1,
  signal: null,
  cmd: 'git rev-list --all --count > _compiled/about/commit-count.txt' }
[15:43:30] Finished 'sheets' after 187 ms

After install:

npm run electron
[1908:0811/154409.371:INFO:CONSOLE(290)] "Uncaught TypeError: Cannot read property 'submenu' of undefined", source: file:///C:/Users/user.name/Downloads/orchestra-master/_compiled/ui/menu.js (290)
[1908:0811/154409.407:INFO:CONSOLE(38)] "
%cOrchestra Studio %cΓÇó Copyright 2016-present, Kary Foundation, Inc.

", source: file:///C:/Users/user.name/Downloads/orchestra-master/_compiled/ui/window.js (38)
(node:1908) DeprecationWarning: Calling an asynchronous function without callback is deprecated.
[1908:0811/154409.551:INFO:CONSOLE(128)] "Uncaught TypeError: Assignment to constant variable.", source: file:///C:/Users/user.name/Downloads/orchestra-master/_compiled/ui/playground.js (128)

I get a nice blank screen with the Kary logo, and some nice, but nonfunctional stoplight buttons at the top, but no functional editor.
node version 6.11.2
npm version 3.10.10

Whitespace problem

On whitespace with only one element checked you get nothing:

Version 1.0 Remaining Features for the milestore

What is not complete for the first version:

Orchestra

• About needs to be refined
• Playground needs to be fixed
  • If possible better editor (monaco really is not designed for this)
  • bigger / smaller font sizes
  • sidebar list of matches
• Ribbon supporting all buttons
  • night mode
  • window controls
• Quick Tests
• Tutorial Reference Page
• Language Reference
  • Fully refined for Orchestra 1.0 language
  • Sidebar TOC be fixed
• File format to become stable and freeze for first version
  • import only compatible files
  • having file version
• Extension Server
  • implement a right protocol for communication

Concerto

• Spaces must be fixed

Orchestra Connector

• For the first version connector must be there for the Visual Studio Code

Build fails on Debian

When I run yarn run build-linux I get the following result:

yarn run v1.3.2
$ gulp --pack --installers --linux --production-build
[21:20:58] Using gulpfile ~/workspace/projects/Orchestra/gulpfile.js
[21:20:58] Starting 'copyResourceFiles'...
[21:20:58] Finished 'copyResourceFiles' after 2.17 ms
[21:20:58] Starting 'sheets'...
Compiling less failed TypeError: this.parse is not a function
[21:20:58] Finished 'sheets' after 1.27 ms
[21:20:58] Starting 'pack-orchestra'...
[21:21:59] Finished 'pack-orchestra' after 1 min
[21:21:59] Starting 'build-orchestra'...
[21:21:59] Finished 'build-orchestra' after 53 μs
[21:21:59] Starting 'default'...
[21:21:59] Finished 'default' after 34 μs
Done in 61.98s.

Error at building for Linux

I'm trying to build it using build document on Linux.
npm run configure and npm install work well. but when in try to run npm run build-linux it failed and show an error to me:

> [email protected] build-linux /home/amirhossein/temp/Orchestra
> gulp --pack --installers --linux --production-build

assert.js:337
throw err;
^

AssertionError [ERR_ASSERTION]: Task function must be specified
at Gulp.set [as _setTask] (/home/amirhossein/temp/Orchestra/node_modules/undertaker/lib/set-task.js:10:3)
at Gulp.task (/home/amirhossein/temp/Orchestra/node_modules/undertaker/lib/task.js:13:8)
at Object. (/home/amirhossein/temp/Orchestra/gulpfile.js:369:10)
at Module._compile (internal/modules/cjs/loader.js:689:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
at Module.load (internal/modules/cjs/loader.js:599:32)
at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
at Function.Module._load (internal/modules/cjs/loader.js:530:3)
at Module.require (internal/modules/cjs/loader.js:637:17)
at require (internal/modules/cjs/helpers.js:20:18)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build-linux: gulp --pack --installers --linux --production-build
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build-linux script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /home/amirhossein/.npm/_logs/2018-09-13T07_44_17_405Z-debug.log

the log file is:

0 info it worked if it ends with ok
1 verbose cli [ '/home/amirhossein/.nvm/versions/node/v10.9.0/bin/node',
1 verbose cli '/home/amirhossein/.nvm/versions/node/v10.9.0/bin/npm',
1 verbose cli 'run',
1 verbose cli 'build-linux' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prebuild-linux', 'build-linux', 'postbuild-linux' ]
5 info lifecycle [email protected]prebuild-linux: [email protected]
6 info lifecycle [email protected]build-linux: [email protected]
7 verbose lifecycle [email protected]build-linux: unsafe-perm in lifecycle true
8 verbose lifecycle [email protected]build-linux: PATH: /home/amirhossein/.nvm/versions/node/v10.9.0/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/home/amirhossein/temp/Orchestra/node_modules/.bin:/home/amirhossein/.nvm/versions/node/v10.9.0/bin:/home/amirhossein/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
9 verbose lifecycle [email protected]build-linux: CWD: /home/amirhossein/temp/Orchestra
10 silly lifecycle [email protected]build-linux: Args: [ '-c', 'gulp --pack --installers --linux --production-build' ]
11 silly lifecycle [email protected]build-linux: Returned: code: 1 signal: null
12 info lifecycle [email protected]build-linux: Failed to exec build-linux script
13 verbose stack Error: [email protected] build-linux: gulp --pack --installers --linux --production-build
13 verbose stack Exit status 1
13 verbose stack at EventEmitter. (/home/amirhossein/.nvm/versions/node/v10.9.0/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:304:16)
13 verbose stack at EventEmitter.emit (events.js:182:13)
13 verbose stack at ChildProcess. (/home/amirhossein/.nvm/versions/node/v10.9.0/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack at ChildProcess.emit (events.js:182:13)
13 verbose stack at maybeClose (internal/child_process.js:961:16)
13 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:250:5)
14 verbose pkgid [email protected]
15 verbose cwd /home/amirhossein/temp/Orchestra
16 verbose Linux 4.15.0-34-generic
17 verbose argv "/home/amirhossein/.nvm/versions/node/v10.9.0/bin/node" "/home/amirhossein/.nvm/versions/node/v10.9.0/bin/npm" "run" "build-linux"
18 verbose node v10.9.0
19 verbose npm v6.2.0
20 error code ELIFECYCLE
21 error errno 1
22 error [email protected] build-linux: gulp --pack --installers --linux --production-build
22 error Exit status 1
23 error Failed at the [email protected] build-linux script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]

I'm using node version v10.9.0 and npm version 6.2.0
thanks Pouya

There must be no menubar on Linux and Windows.

Orchestra's Menu bar must be only visible on the macOS. On Windows and Linux the menubar will be augmented to the title less window and destroyed the whole UI.

Therefore these steps are needed:

• Disabling menubar on Windows and Linux
• Adding a Keyboard-Event Manager to handle the shortcuts without using menubar.
• Adding all the missing buttons of the menubar to the UI5's ribbon.

Concerto: & renders horrible

When you compile &, it loads as:

Funny is that & actually compiles to a XML safe &

Concerto: Detecting \s wrong

Alphabet system

Each alphabet contains a set of ranges and other stuff that can be either wildcard special characters or plain text. To overcome this there must be a fine way to use alphabets.

The Sigma block now provides a custom "more free" alphabet maker. But I think there must be a special range block for the Sigma

Concerto/Test Case Failure

/((['"])(?:(?!\2|\\).|\\(?:\r\n|[\s\S]))*(\2)?|`(?:[^`\\$]|\\[\s\S]|\$(?!\{)|\$\{(?:[^{}]|\{[^}]*\}?)*\}?)*(`)?)|(\/\/.*)|(\/\*(?:[^*]|\*(?!\/))*(\*\/)?)|(\/(?!\*)(?:\[(?:(?![\]\\]).|\\.)*\]|(?![\/\]\\]).|\\.)+\/(?:(?!\s*(?:\b|[\u0080-\uFFFF$\\'"~({]|[+\-!](?!=)|\.?\d))|[gmiyu]{1,5}\b(?![\u0080-\uFFFF$\\]|\s*(?:[+\-*%&|^<>!=?({]|\/(?![\/*])))))|(0[xX][\da-fA-F]+|0[oO][0-7]+|0[bB][01]+|(?:\d*\.\d+|\d+\.?)(?:[eE][+-]?\d+)?)|((?!\d)(?:(?!\s)[$\w\u0080-\uFFFF]|\\u[\da-fA-F]{4}|\\u\{[\da-fA-F]+\})+)|(--|\+\+|&&|\|\||=>|\.{3}|(?:[+\-\/%&|^]|\*{1,2}|<{1,2}|>{1,3}|!=?|={1,2})=?|[?~.,:;[\](){}])|(\s+)|(^$|[\s\S])/g

Missing special characters

Quartet is missing these special characters

• `\b' --> word boundary
• `\B' --> not word boundary
• \w --> word