Giter Site home page Giter Site logo

node-python-bridge's Introduction

python-bridge Build Status Build Status

Most robust and simple Python bridge. Features, and comparisons to other Python bridges below, supports Windows.

API

View documentation with TypeScript examples.

npm install python-bridge
'use strict';

let assert = require('assert');
let pythonBridge = require('python-bridge');

let python = pythonBridge();

python.ex`import math`;
python`math.sqrt(9)`.then(x => assert.equal(x, 3));

let list = [3, 4, 2, 1];
python`sorted(${list})`.then(x => assert.deepEqual(x, list.sort()));

python.end();

var python = pythonBridge(options)

Spawns a Python interpreter, exposing a bridge to the running processing. Configurable via options.

  • options.python - Python interpreter, defaults to python

Also inherits the following from child_process.spawn([options]).

  • options.cwd - String Current working directory of the child process
  • options.env - Object Environment key-value pairs
  • options.stdio - Array Child's stdio configuration. Defaults to ['pipe', process.stdout, process.stderr]
  • options.uid - Number Sets the user identity of the process.
  • options.gid - Number Sets the group identity of the process.
var python = pythonBridge({
    python: 'python3',
    env: {PYTHONPATH: '/foo/bar'}
});

python`expression(args...)`.then(...)

Evaluates Python code, returning the value back to Node.

// Interpolates arguments using JSON serialization.
python`sorted(${[6, 4, 1, 3]})`.then(x => assert.deepEqual(x, [1, 3, 4, 6]));

// Passing key-value arguments
let obj = {hello: 'world', foo: 'bar'};
python`dict(baz=123, **${obj})`.then(x => {
    assert.deepEqual(x, {baz: 123, hello: 'world', foo: 'bar'});
});

python.ex`statement`.then(...)

Execute Python statements.

let a = 123, b = 321;
python.ex`
    def hello(a, b):
        return a + b
`;
python`hello(${a}, ${b})`.then(x => assert.equal(x, a + b));

python.lock(...).then(...)

Locks access to the Python interpreter so code can be executed atomically. If possible, it's recommend to define a function in Python to handle atomicity.

python.lock(python => {
    python.ex`hello = 123`;
    return python`hello + 321'`;
}).then(x => assert.equal(x, 444));

// Recommended to define function in Python
python.ex`
    def atomic():
        hello = 123
        return hello + 321
`;
python`atomic()`.then(x => assert.equal(x, 444));

python.stdin, python.stdout, python.stderr

Pipes going into the Python process, separate from execution & evaluation. This can be used to stream data between processes, without buffering.

let Promise = require('bluebird');
let fs = Promise.promisifyAll(require('fs'));

let fileWriter = fs.createWriteStream('output.txt');

python.stdout.pipe(fileWriter);

// listen on Python process's stdout
python.ex`
    import sys
    for line in sys.stdin:
        sys.stdout.write(line)
        sys.stdout.flush()
`.then(function () {
    fileWriter.end();
    fs.readFileAsync('output.txt', {encoding: 'utf8'}).then(x => assert.equal(x, 'hello\nworld\n'));
});

// write to Python process's stdin
python.stdin.write('hello\n');
setTimeout(() => {
    python.stdin.write('world\n');
    python.stdin.end();
}, 10);

python.end()

Stops accepting new Python commands, and waits for queue to finish then gracefully closes the Python process.

python.disconnect()

Alias to python.end()

python.kill([signal])

Send signal to Python process, same as child_process child.kill.

let Promise = require('bluebird');

python.ex`
    from time import sleep
    sleep(9000)
`.timeout(100).then(x => {
    assert.ok(false);
}).catch(Promise.TimeoutError, (exit_code) => {
    console.error('Python process taking too long, restarted.');
    python.kill('SIGKILL');
    python = pythonBridge();
});

Handling Exceptions

We can use Bluebird's promise.catch(...) catch handler in combination with Python's typed Exceptions to make exception handling easy.

python.Exception

Catch any raised Python exception.

python.ex`
    hello = 123
    print(hello + world)
    world = 321
`.catch(python.Exception, () => console.log('Woops!  `world` was used before it was defined.'));

python.isException(name)

Catch a Python exception matching the passed name.

function pyDivide(numerator, denominator) {
    return python`${numerator} / ${denominator}`
        .catch(python.isException('ZeroDivisionError'), () => Promise.resolve(Infinity));
}
pyDivide(1, 0).then(x => {
    assert.equal(x, Infinity);
    assert.equal(1 / 0, Infinity);
});

pythonBridge.PythonException

Alias to python.Exception, this is useful if you want to import the function to at the root of the module.

pythonBridge.isPythonException

Alias to python.isException, this is useful if you want to import the function to at the root of the module.


Features

  • Does not affect Python's stdin, stdout, or stderr pipes.
  • Exception stack traces forwarded to Node for easy debugging.
  • Python 2 & 3 support, end-to-end tested.
  • Windows support, end-to-end tested.
  • Command queueing, with promises.
  • Long running Python sessions.
  • ES6 template tags for easy interpolation & multiline code.

Comparisons

After evaluating of the existing landscape of Python bridges, the following issues are why python-bridge was built.

  • python-shell — No promises for queued requests; broken evaluation parser; conflates evaluation and stdout; complex configuration.
  • python — Broken evaluation parsing; no exception handling; conflates evaluation, stdout, and stderr.
  • node-python — Complects execution protocol with incomplete Python embedded DSL.
  • python-runner — No long running sessions; child_process.spawn wrapper with unintuitive API; no serialization.
  • python.js — Embeds specific version of CPython; requires compiler and CPython dev packages; incomplete Python embedded DSL.
  • cpython — Complects execution protocol with incomplete Python embedded DSL.
  • eval.py — Can only evaluate single line expressions.
  • py.js — For setting up virtualenvs only.

License

MIT

node-python-bridge's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

node-python-bridge's Issues

How to keep python bridge connected?

Hi,

Thanks for making this tool!

I'm new to web development (and not familiar with Python), and am trying to use a Python package to scrape and return website data to a JS object.

The Python package takes a url and returns an object. That object can then use methods to pull data from the object (and save that data to a variable, for example)

I'm having an issue when trying to call multiple methods using async/await. After calling the first method, python bridge is no longer connected, and errors out when trying to call the second method (PythonBridgeNotConnected).

I'm assuming this is working correctly and that I'm not fully understanding the package, but how can I use python bridge without it disconnecting like this?

import pythonBridge from 'python-bridge'
let python = pythonBridge()
const url = 'some_url'
python.ex`from scrapers import scrape_me`
python.ex`scraper = `scrape_me(${url})`

let jsObject = {}

async function pyscript() {
  try {
    // this runs correctly
    jsObject.name = await python`scraper.title()`
    // this errors out
    jsObject.instructions = await python`scraper.instructions_list()`
  } catch (error) {
    console.log(error)
  }
}

(async () => {
  await pyscript()
})().catch(error => {
  console.log('error')
  console.error(error)
})

python.end()

Python-Bridge not working after app is built

Python-bridge works fine in the dev environment, but once I package the app using electron-builder it breaks. This seems to be related to ASAR. Node.js is able to properly read ASAR paths, but python is not. Note the app.asar in the path. It would be much appreciated if someone knows how to fix this. Until then I will try to exclude the file from ASAR.

This is the output when I run my packaged app on the command line:

C:\Program Files\binpro\resources\assets\WinPython\python.exe: can't open file 'C:\\Program Files\\binpro\\resources\\app.asar\\node_modules\\python-bridge\\node_python_bridge.py': [Errno 2] No such file or directory

How to execute py file through node-python-bridge?

Hi,

I have a requirement to execute a py file through my node service, i earlier used https://github.com/extrabacon/python-shell to execute py files but it doesn't have good error handling and option to open bridges.
I was going through node-python-bridge and it looks good to me to execute python code but i am not able to execute py files through it.

Is there any way to execute py files?
can we make line

let ps = child_process.spawn(intepreter, [PYTHON_BRIDGE_SCRIPT], options);
more dynamic to accept user input python file?

Can't import modules

Hi,

Great project! I was wondering if it is still alive?

My issue is that when I define

var python = pythonBridge({
    python: '/anaconda3/bin/python',
    env: {PYTHONPATH: 'valid/pythonpath'}
});

then I can not import common modules, for instance,numpy.

Allow passing arbitrary arguments to the python interpreter

let ps = child_process.spawn(intepreter, [PYTHON_BRIDGE_SCRIPT], options);

Issue

Hardcoding [PYTHON_BRIDGE_SCRIPT] here prevents users from passing any arguments to the interpreter.

My use case

I want to pass -u to python3 in order to flush stdout after every print.
This way I don't have to manually flush the stdout after (via sys.stdout.flush()) or within (via print(message, flush=True) every print.

Related Issues

#35

Warning: Promise.defer is deprecated... Use new Promise instead.

Hi there,
So, grad school started, I have profs wanting assignments turned in using Python; but I'm accustomed to programming in Meteor/Node. So, I'm trying to integrate node-python-bridge into a Meteor app. Run algorithms with Python, display on a reactive Meteor app. Sounds like that could be a useful tool, right? Well it does to me, anyhow.

Environment
So, the environment I'm using is as follows:

  • Meteor: v1.4.1.1
  • Node: v2.15.9
  • Node-Gyp: v3.4.0
  • Mac: 10.11.5 (El Capitan)

Initial Error
Got the initial Meteor app set up; installed node-python-bridge, but am getting the following error from Line 137 when I run the app:

W20160920-19:08:18.851(-5)? (STDERR) Warning: Promise.defer is deprecated and will be removed in a future version. Use new Promise instead.
W20160920-19:08:18.944(-5)? (STDERR)     at enqueue (/Users/abigail/Code/meteor-python/node_modules/python-bridge/index.js:137:29)
W20160920-19:08:18.945(-5)? (STDERR)     at Function.wrapper [as ex] (/Users/abigail/Code/meteor-python/node_modules/python-bridge/index.js:35:20)
W20160920-19:08:18.945(-5)? (STDERR)     at meteorInstall.server.main.js (server/main.js:10:8)
W20160920-19:08:18.946(-5)? (STDERR)     at fileEvaluate (packages/modules-runtime/.npm/package/node_modules/install/install.js:153:1)
W20160920-19:08:18.946(-5)? (STDERR)     at require (packages/modules-runtime/.npm/package/node_modules/install/install.js:82:1)
W20160920-19:08:18.946(-5)? (STDERR)     at /Users/abigail/Code/meteor-python/.meteor/local/build/programs/server/app/app.js:38:1
W20160920-19:08:18.947(-5)? (STDERR)     at /Users/abigail/Code/meteor-python/.meteor/local/build/programs/server/boot.js:292:10
W20160920-19:08:18.947(-5)? (STDERR)     at Array.forEach (native)
W20160920-19:08:18.948(-5)? (STDERR)     at Function._.each._.forEach (/Users/abigail/.meteor/packages/meteor-tool/.1.4.1_1.1h0re2h++os.osx.x86_64+web.browser+web.cordova/mt-os.osx.x86_64/dev_bundle/server-lib/node_modules/underscore/underscore.js:79:11)
W20160920-19:08:18.948(-5)? (STDERR)     at /Users/abigail/Code/meteor-python/.meteor/local/build/programs/server/boot.js:128:5

Attempted Fix
I've not entirely learned the new Promise syntax; so I thought this would be a good time to try to familiarize myself with it. This is the fix that I came up with:

// Only a single Python worker process can be asynchronously running at a time,
// which means that only a single python _function_ can be running at a time.
// If a new worker function is queued up; it forces the bridge to try to resolve
// the last worker process.

function singleQueue() {
  return function enqueue(f) {
    return new Promise(function(resolve, reject){
      Promise.try(f).then(resolve, reject);
    });
  };
}


// function singleQueue() {
//     let last = Promise.resolve();
//     return function enqueue(f) {
//         let defer = Promise.defer();
//         let wait = last;
//         last = defer.promise;
//         return new Promise((resolve, reject) => {
//             wait.finally(() => {
//                 Promise.try(f).then(resolve, reject);
//             });
//         }).finally(() => {
//             defer.resolve();
//         });
//     };
// }

Subsequent Error
This produces an IOError: [Errno 32] Broken pipe error, which I think is a good sign. Or I might be walking off a cliff here. I'm not sure which.

[[[[[ ~/Code/Biomedical/meteor-python ]]]]]   

=> Started proxy.                             
=> Started MongoDB.                           
=> Started your app.                          

=> App running at: http://localhost:3000/
W20160920-19:31:23.361(-5)? (STDERR) Traceback (most recent call last):
W20160920-19:31:23.428(-5)? (STDERR)   File "/Users/abigail/Code/meteor-python/node_modules/python-bridge/node_python_bridge.py", line 61, in <module>
W20160920-19:31:23.428(-5)? (STDERR)     writer.flush()
W20160920-19:31:23.429(-5)? (STDERR) IOError: [Errno 32] Broken pipe
W20160920-19:31:23.429(-5)? (STDERR) close failed in file object destructor:
W20160920-19:31:23.429(-5)? (STDERR) IOError: [Errno 9] Bad file descriptor

Any ideas if I'm on the right path here? Does that singleQueue() function look correct as it's rewritten?

Thanks in advance for taking a look at this problem!

Boolean object property in Python function parameter

var s = {
    a: "asd",
    b: 1,
    c: true,
};

let rtn = await python`print(${s})`;

The s.c parameter leads to an exception thrown. Strings/numbers work, arrays as well. Only boolean and null values (including NaN) throw errors:

(node:4232) UnhandledPromiseRejectionWarning: Error: Traceback (most recent call last):
  File "...\node_modules\python-bridge\node_python_bridge.py", line 94, in <module>
    value = eval(_compile(data['code'], '<input>', 'eval'), _locals)
  File "<input>", line 1, in <module>
NameError: name 'true' is not defined

The easiest solution I found was to adapt _locals, but I have no idea if that leads to other problems (maybe also for Infinity/NaN?):

_locals = {'__name__': '__console__', '__doc__': None, 'true': True, 'false': False, 'null': None}

The cleaner solution could be to use json.loads to convert JSON input strings to a Python dict?

Node.js is broken when python code returns NaN

const python = pythonBridge();
python`float('nan')`.then(n => console.log(n))

This sent incorrect message to channel and causes unhandleable error:

undefined:1
{"type":"success","value":NaN}
                          ^

SyntaxError: Unexpected token N in JSON at position 26
    at JSON.parse (<anonymous>)
    at Pipe.channel.onread (internal/child_process.js:472:28)

If value contains NaN we'll get the same error (python`{"a": float('nan')}`.then(n => console.log(n))).

Regex support using rfind()?

When a value is '\n', I get "EOL while scanning string literal" while trying to use rfind(). Does Python-Bridge support regex in this manner? Apologies if this question is easily answered. I am new to JavaScript programming.

stdout/stderr are null?

Hello,

When I do the following:

let pythonBridge = require('python-bridge');
let python = pythonBridge();

python.stdin contains an object but stdout and stderr are null - maybe due to something else unrelated

However I think this is mainly because of the default options for stdio. So I tried passing in 'pipe' into 2nd arg to options.stdio on startup such as below:

let python = pythonBridge({  stdio: ['pipe','pipe', process.stderr] });
python.stdout.pipe(someStream);

Now stdout does seem to work better so I am guessing 'pipe' is a magic word somewhere that makes things work. I am on OSX running Python 3.

Python-Bridge not working after create package using pkg npm

When I am creating the package using the pkg npm, it is giving below error while running the application.

  • python: can't open file '/snapshot/home/jay/project_dir/project_folder/modules/python-bridge/node_python_bridge.py': [Errno 2] No such file or directory

Also, the python bridge folder and node_python_bridge.py are present in the both module and node modules directory.

Assertion failed: ipc_frame.header.flags

trying to run the example in the readme, I get this...

Assertion failed: ipc_frame.header.flags <= (UV_IPC_TCP_SERVER | UV_IPC_RAW_DATA | UV_IPC_TCP_CONNECTION), file src\win\pipe.c, line 1591

EOL error when "\n" in python script

Hi! First of all it is a wonderful package that allows me to call python script from Node.js super fast and easy. There is one problem that I encountered during the work though.

In side my python script I need to return a string that requires a "\n" in side. For example:
return "Path: " + path + "\nVersion: " + version

But when there is this new line, the EOL error will appear:

return "Path: " + path + "
^
SyntaxError: EOL while scanning string literal

(the ^ is right below the ")

So basically the system cannot read the "\n" and take it as the end of the string.

After I removed the "\n", no more EOL error and the script runs smoothly. But the output becomes messy as it is hard to read without the line break.

May I know whether this issue does exist, and whether there is any solution or work-arounds?

thank you for the excellent work btw!

A more detailed error msg in case needed:

stack trace: Error: Traceback (most recent call last):
File "c:\Users*\node_modules\python-bridge\node_python_bridge.py", line 101, in
_exec(_compile(data['code'], '', 'exec'), _locals)
File "C:\Users*
\AppData\Local\Programs\Python\Python39\lib\codeop.py", line 143, in call
codeob = compile(source, filename, symbol, self.flags, True)
File "", line 8
return "Path: " + path + "
^
SyntaxError: EOL while scanning string literal
at ChildProcess.onMessage (c:\Users***\node_modules\python-bridge\index.js:44:32)
at Object.onceWrapper (events.js:422:26)
at ChildProcess.emit (events.js:315:20)
at emit (internal/child_process.js:903:12)
at processTicksAndRejections (internal/process/task_queues.js:81:21)

Python dependency on six

Issue

node-python-bridge has a hard dependency on python package six. This is not documented and fails when trying to run. See error below.

engine    | error: Pipeline: Failed render_message1undefined
engine    | Traceback (most recent call last):
engine    |   File "/usr/src/app/node_modules/python-bridge/node_python_bridge.py", line 3, in <module>
engine    |     import six
engine    | ModuleNotFoundError: No module named 'six'

Solution

I'm curious if there is anyway to remove that dependency and only use native functions. In the short term, I think we need to document this in the README for others.

Allow using native promises?

Hi,

First of all: I really like the way you wrote this bridge - I was on my way of writing the same thing on my own so this is seriously cool to see. :)

I see that your code depends on bluebird - which is quite heavy when compared to native promises. It has the upshot of being available anywhere, but perhaps you'd consider moving to native promises with some polyfill like any-promise and add bluebird as optional dependency?

I would be happy to help.

Typing bug: Argument of type '{ python: string; }' is not assignable to parameter of type 'PythonBridgeOptions | undefined'.

The following code:

  const python = pythonBridge({
    python: 'python3',
  })

Will generate the following TSLint error:

[ts]
Argument of type '{ python: string; }' is not assignable to parameter of type 'PythonBridgeOptions | undefined'.
  Object literal may only specify known properties, and 'python' does not exist in type 'PythonBridgeOptions | undefined'.

Which is caused by the following declaration:

export interface PythonBridgeOptions {
-  intepreter?: string;
+  python?: string;
  stdio?: [PipeStdin, PipeStdout, PipeStderr];
  cwd?: string;
  env?: { [key:string]: string; };
  uid?: number;
  gid?: number;
}

PythonBridgeOptions not compatible with Node.js

According to the Node.js types:

    export interface ProcessEnv {
      [key: string]: string | undefined
    }

We need to compatible with it by:

-  env?: { [key:string]: string; };
+  env?: { [key: string]: string | undefined; };

'generator' object has no attribute '__dict__

When I am calling GroupkFold.split()
Internally it is calling python bridge

I am getting an error in python bridge:

error: undefined,
exception: {
type: { name: 'AttributeError', module: 'builtins' },
message: "'generator' object has no attribute 'dict'",
args: [ "'generator' object has no attribute 'dict'" ],
format: [
"AttributeError: 'generator' object has no attribute 'dict'\n"
]

traceback: {
lineno: 111,
strack: [
' File "node_modules@fisch0920\python-bridge\node_python_bridge.py", line 114, in \n' +
" response = dict(type='exception', value=format_exception(t, e, tb))\n"
],
format: [
' File "..\node_modules@fisch0920\python-bridge\node_python_bridge.py", line 111, in \n' +
" response = dict(type='success', value=json.dumps(value, separators=(',', ':'), cls=JavaScriptEncoder))\n",
' File "Programs\Python\Python310\lib\json\init.py", line 238, in dumps\n' +
' **kw).encode(obj)\n',
' File "Programs\Python\Python310\lib\json\encoder.py", line 199, in encode\n' +
' chunks = self.iterencode(o, _one_shot=True)\n',
' File "Python\Python310\lib\json\encoder.py", line 257, in iterencode\n'
+
' return _iterencode(o, 0)\n',
' File "node_modules@fisch0920\python-bridge\node_python_bridge.py", line 81, in default\n' +
' return o.dict\n'
]
},

I am confused why this error is triggered.

TypeScript Defination Support?

Hi, @mickdekkers !

Thanks for your great project, I'm planning to use node-python-bridge to call TensorFlow in Node.js.

Because I'm a big TypeScript fan, so I'd like to suggest adding a typing definition file in this project, which could benefit us from the static type checking and vscode IntelliSense from TypeScript.

Or be better, rewrite the index.js in TypeScript.

If you agree with that, I'd be very like to contribute.

ModuleNotFound exceptions

I'm trying to import a library from the working directory but I get a ModuleNotFound exception. I've been looking at the code and I think it might be related to this line: https://github.com/Submersible/node-python-bridge/blob/master/node_python_bridge.py#L91
Since there are no globals defined, the import system isn't working correctly. Is there a way to fix this?

let pythonBridge = require('python-bridge');

let python = pythonBridge({
    cwd: process.cwd()
});

python.ex`import mylibrary`;
python`x = mylibrary.helloWorld()`.then(x => {
    console.log(x)
});

python.end();

run a python file ?

How can I run a python file with node-python-bridge instead of directly writing all of my code in a javascript string ?

python-shell has a broken evaluation parser?

In your readme you say that:

python-shell — No promises for queued requests; broken evaluation parser; conflates evaluation and stdout; complex configuration.

what do you mean by broken evaluation parser?

python.ex`${src}` is broken

I can't do python.ex${src} for some reason. Though, it's problem with node js.

Is there any chance we are getting python.ex2('something here') also?

Support for running in worker threads

With the default options the following doesn't work.

const python = pythonBridge();

Instead we get the following error. If I set stdio to ['ignore', 'pipe', 'pipe'] it works without an issue. It'd be nice to either detect the worker thread and use different defaults or catch the error and show a nicer message.

TypeError [ERR_INVALID_OPT_VALUE]: The value "WritableWorkerStdio {
  _writableState: WritableState {
    objectMode: false,
    highWaterMark: 16384,
    finalCalled: false,
    needDrain: false,
    ending: false,
    ended: false,
    finished: false,
    destroyed: false,
    decodeStrings: false,
    defaultEncoding: 'utf8',
    length: 97,
    writing: true,
    corked: 0,
    sync: false,
    bufferProcessing: false,
    onwrite: [Function: bound onwrite],
    writecb: [Function],
    writelen: 97,
    bufferedRequest: null,
    lastBufferedRequest: null,
    pendingcb: 1,
    prefinished: false,
    errorEmitted: false,
    emitClose: true,
    autoDestroy: false,
    bufferedRequestCount: 0,
    corkedRequestsFree: {
      next: null,
      entry: null,
      finish: [Function: bound onCorkedFinish]
    }
  },
  writable: true,
  _events: [Object: null prototype] {},
  _eventsCount: 0,
  _maxListeners: undefined,
  [Symbol(kPort)]: MessagePort {
    active: true,
    refed: true,
    _events: [Object: null prototype] {
      newListener: [Function],
      removeListener: [Function],
      message: [Function]
    },
    _eventsCount: 3,
    [Symbol(kWaitingStreams)]: 1
  },
  [Symbol(kName)]: 'stdout',
  [Symbol(kWritableCallbacks)]: [ [Function: bound onwrite] ]
}" is invalid for option "stdio"
    at internal/child_process.js:996:13
    at Array.reduce (<anonymous>)
    at getValidStdio (internal/child_process.js:922:17)
    at ChildProcess.spawn (internal/child_process.js:340:11)
    at Object.spawn (child_process.js:535:9)
    at Function.pythonBridge (/Users/xo/code/scratch/node_modules/python-bridge/index.js:22:28)
    at PostProcessor.<anonymous> (/Users/xo/code/scratch/workers/post-processor.ts:91:24)
    at step (/Users/xo/code/scratch/workers/post-processor.ts:50:23)
    at Object.next (/Users/xo/code/scratch/workers/post-processor.ts:31:53)
    at /Users/xo/code/scratch/workers/post-processor.ts:25:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/xo/code/scratch/workers/post-processor.ts:21:12)
    at PostProcessor.guessit (/Users/xo/code/scratch/workers/post-processor.ts:96:16)
    at /Users/xo/code/scratch/workers/post-processor.ts:78:29
    at Array.map (<anonymous>)
    at PostProcessor.<anonymous> (/Users/xo/code/scratch/workers/post-processor.ts:77:36)

Optionally importing unicode_literals from __future__

Hi,

Frist, thank you! Great work!

While working with some legacy libraries I sometimes unicode issues due to the use of unicode_literals

In one of the libraries we use object (dict) keys are used by the library as XPath path.
I did not see the internal implementation but I assume some logic is applied to tokenize, parse and process the "path" and the library does not expect to get a unicode string there and does not handle them.

When node_python_bridge.py does:

from __future__ import unicode_literals

It breaks.

For example, if I send the key default

root.something['defualt']

I will get an error:

badly formatted or nonexistent path (8): Bad key "d e f a u l t"

The library handled the unicode bytes as string without converting first, which somehow ended up with a space between every char pair.

I know that the problem is in the library I use and not in this library, but can it be set to optionally import unicode_literals, based on some option parameter?

Thanks!

For reference: http://python-future.org/unicode_literals.html

Purpose: venv & modules management support

When I'm developing my FaceNet project with python-bridge, I use venv to get an isolated Python environment, inside my project directory.

The reason to do this is that I want to do an Electron Package in the future, so I need all the python modules be there out-of-the-box. The venv directory includes the Tensorflow, Numpy, etc, cost about 500MB storage size.

Today, I'm thinking of if I should split the different neural network models to different NPM module, like: facenet for Google FaceNet, chinese-whisper for the Chinese Whisper Althorithm, `Mtcnn for the MTCNN model.

However, it turns out that if I use the current method to create those npm modules, they all will create a Python venv by themselves, then there will be lots of venv directories inside themselves, cause storage waste.

My purpose is: manage the venv by python-bridge, install all Python modules inside a venv directory of python-bridge.

Then, all other modules can share the venv directory via python-bridge, instead of creating & install by themselves.

The Code

API might look like:

python = pythonBridge({ venv: 'dir' })
if (!await python`'tensorflow' in sys.modules`) {
  await python.pip.install('tensorflow')
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.