Comments (21)
Ahh I closed the issue too early. We ended up splitting some of the work to address this into CLIPPER-143 which hasn't been completed yet.
from clipper.
You are correct. This is currently the case. We have an open JIRA issue CLIPPER-107 to address this.
It's a fairly high priority for us. I expect we will get to it sometime this week.
from clipper.
Cool. Do you have an ETA for this feature? We are eager to use clipper in our product :)
from clipper.
JSON requests are currently supported. You can specify containers that expect strings as input, which can of course be JSON strings. And yes ultimately all inputs come to Clipper as JSON through the REST interface.
The issue under discussion was being able to return arbitrary JSON from containers, which is not currently supported but will be shortly.
from clipper.
This is now under review: #134
from clipper.
@frank2wang87 That's presently the case. y_hat_
will be a string (json-formatted or otherwise) after #134 is merged.
from clipper.
Fixed by #116
from clipper.
How does this work? I still cant return a string from my container
def predict_strings(self, inputs):
print ('received input', inputs)
return 'Hello'
It seems like the output from query_frontend.hpp
is casted as float
Line 289
clipper::json::add_double(json_response, PREDICTION_RESPONSE_KEY_OUTPUT,
query_response.output_.y_hat_);
Am i doing something wrong? I created an app which accepts strings
and i want to output a custom json.
from clipper.
It seems that the image should be translated to a list to generate a json request.(Line 123 in cifar_utils.py)
req_json = json.dumps({'uid': uid, 'input': list(x)})
The list is then translated to original image if the model container receive the true image as input. It is low efficiency when disposing large image or video. Correct me if I am wrong.
Does the clipper support REST input? i.e, does clipper receive original image(or video, or texture) as input and transfer it to the docker container?
from clipper.
When will this feature be published? We are eager to use it :)
from clipper.
@Corey-Zumar what's the status of this?
from clipper.
@dcrankshaw In progress - continuing to work on it tonight and tomorrow
from clipper.
L389 in query_frontend.hpp
clipper::json::add_double(json_response, PREDICTION_RESPONSE_KEY_OUTPUT, query_response.output_.y_hat_);
Now it still returns a double output(double y_hat_). Am I right ?
from clipper.
Cool.
from clipper.
Has this issue been solved yet? When will the 0.1 alpha be released? We are looking forward to it.
from clipper.
Waiting on #134 to be merged. IMO, the change is stable. You can merge the branch associated with #134 into your copy of the repo and start experimenting with the functionality.
from clipper.
Any ETA on when this is going to be merged?
from clipper.
Just merged. I was out of town traveling for a few days which caused a delay.
from clipper.
I am still not able to send a simple JSON using predict_strings.
The following code
def predict_strings(self, inputs):
return json.dumps(['foo', {'bar': ('baz')}])
gives the output
{"query_id":13,"output":"\u0001","default":false}
Am i doing anything wrong here?
from clipper.
Hmm let me dig into this. I'll report back soon.
from clipper.
Ahh the problem is how you are formulating the JSON response. The input to the predict function is a list of strings and Clipper expects you to return a list of JSON responses, one for each input. Try this:
def predict_strings(self, inputs):
outputs = []
for i in inputs:
outputs.append(json.dumps(['foo', {'bar': ('baz')}]))
return outputs
I just added some more explicit type checking in #151 to make this easier to catch.
from clipper.
Related Issues (20)
- Batch_size issue HOT 5
- Metrics
- Debugging input and output
- Number of predictions mismatch
- Potential deadlock when deleting model container replica queue
- Issue with speed of clipper with custom PyTorch model HOT 1
- Model Selection HOT 4
- Study the source code and development
- Dynamic Batching HOT 2
- discussion question HOT 2
- Failed to retrieve a prediction response within the specified latency SLO HOT 2
- No connected models found for query HOT 1
- all input-type (except double) do not work.
- Kubernetes Connection - the server could not find the requested resource
- Question for python_deployer.deploy_python_closure - case of multiple outputs from func
- About the source code HOT 1
- basic query example HOT 1
- develop test failed on win10: qf startup error *Option ‘thread_pool_size’ does not exist*
- Unregistering an application, and then registering again with different output doesn't change the output
- Documentation website unavailable(expired) HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clipper.