Comments (4)
Hi @deep-diver,
If that is the case, the problem is that I want to allocate one GPU k8s node for one specific TFX component.
Last time, I checked TFX didn't support component specific node assignments when KFP is used as an orchestrator. Maybe @rcrowe-google knows if/when TFX supports component specific node assignments.
Regarding GCP AI Platform: Can you share a bit more info around your configuration?
I suspect that the eval is running on a non-GPU node and depending on the number of samples during your eval step, it can take forever. Try to run it on a GPU or switch to ALBERT which executes faster on CPUs (example is in the same Github folder).
I hope this helps.
from workshops.
I am not sure how to set smaller step number?
eval_config = tfma.EvalConfig(
model_specs=[tfma.ModelSpec(label_key='label')],
slicing_specs=[tfma.SlicingSpec()],
metrics_specs=[
tfma.MetricsSpec(metrics=[
tfma.MetricConfig(
class_name='CategoricalAccuracy',
threshold=tfma.MetricThreshold(
value_threshold=tfma.GenericValueThreshold(
lower_bound={'value': 0.5}),
change_threshold=tfma.GenericChangeThreshold(
direction=tfma.MetricDirection.HIGHER_IS_BETTER,
absolute={'value': -1e-2})))
])
]
)
evaluator = Evaluator(
examples=example_gen.outputs['examples'],
model=trainer.outputs['model'],
baseline_model=model_resolver.outputs['model'],
eval_config=eval_config
)
components.append(evaluator)
I will try the ALBERT version! Thanks! 👍 👍
But I want to know if there is a way to allocate GPU machine only for TFX Evaluator component. Since this is a simple personal project, I don't want have multiple k8s GPU nodes. (can I set one node with GPU, and two nodes without GPU?)
from workshops.
Configuring nodes for specific components is in development now, and should be available soon.
@zhitaoli
from workshops.
Or is there a way to wrap TFX component in @func_to_container_op
?
from workshops.
Related Issues (13)
- use lstm
- Notebooks in README moved HOT 2
- Real Time Forecasting/Prediction Not getting feasible
- workshops/extras/archive/05_custom_estimators.ipynb wont render HOT 1
- movie-classification won't work on TF 1.13 or above HOT 1
- Workshops fail with "This file has moved" HOT 2
- Not authorized to get pickles HOT 5
- Worshop "File has moved"
- Could not parse example input in TFX_Pipeline_for_Bert_Preprocessing HOT 10
- https://storage.googleapis.com/tensorflow-workshop-examples/stack-overflow-data.csv missing HOT 4
- Dataflow fails on Transform component in "TFX_Pipeline_for_Bert_Preprocessing" HOT 2
- very confused with tensorflow java api HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from workshops.