Comments (42)
Hello! @nrellanti
Just a quick update! We are almost done with the protobuff support, we had been delayed trying to investigate about external imports in the protobuf file. We still don´t have that feature developed, but we will release the first version of the protobufs around next week (this time for real) and we will try to find a way to develop support for external imports in the near future.
Sorry for all the troubles and delays, it has been a ride for sure! 😅
Cheers!
from kloadgen.
Hello:
I am curious about the progress for adding protobuf support. Is there an expected timeline for completion?
Thank you!
from kloadgen.
We are currently working on a solution. We hope to have it implemented in early May and we will notify you.
from kloadgen.
Yes. Sorry for the inconvenience, we will have it as soon as possible.
from kloadgen.
Hi,
I will try to get these feature release in the end of next month.
If not resolved until then I will update with a new estimation plus the % of work tha reamina to do for these feature.
from kloadgen.
Hi @madiday,
My apologizes for the delays, we had some big problems with this integration and other unrelated issues. But we are close to offer a solution quite soon. Next week I will be able to provide a solution. In the meantime, can you provide a proto file according to you expectation, so we can include it in our testing and be aligned with you?
Thanks and we are really sorry about.
from kloadgen.
Hello @madidaye
@vtorresb and I are currently working on protobuff support, we have just completed the extractor which is about 40% of what we need to do in order to generate random values. I can´t say a specific timeframe where we will finish this, but hopefully we might have a working branch soon.
Again sorry for the delays, we know you have been waiting for too long but hopefully we will good news soon.
from kloadgen.
HI @madidaye ,
just a small update. We are working on it and is possible to get something for you to test close to the end of the month.. may be a bit later, but not too much.
Cheers!!
from kloadgen.
Is this issue still in progress?
from kloadgen.
Is there an estimated date of completion on this feature at this time?
from kloadgen.
Is there a timeframe update on this feature?
from kloadgen.
Thank you for your response.
Here is a reference proto file
syntax = "proto3";
package company;
import "google/protobuf/timestamp.proto";
option java_package = "com.company.finance.proto";
option java_multiple_files = true;
message IncidentEvent {
int32 id = 1;
string occurrence_id = 2;
string load_number = 3;
.company.IncidentEvent.AttributeType claim_type = 4;
.company.IncidentEvent.AttributeType collision_type = 5;
.company.IncidentEvent.AttributeType incident_cause_type = 6;
.company.IncidentEvent.AttributeType incident_type = 7;
.company.IncidentEvent.AttributeType review_status_type = 8;
double incident_latitude = 9;
double incident_longitude = 10;
string incident_date = 11;
.google.protobuf.Timestamp incident_time = 12;
string incident_city = 13;
string incident_state = 14;
string location_description = 15;
repeated .company.IncidentEvent.IncidentEquipment incident_equipment_details = 16;
.company.IncidentEvent.Driver driver = 17;
string dot_accident_indicator = 18;
string drug_test_required_indicator = 19;
string hazardous_material_indicator = 20;
string preventable_indicator = 21;
string report_by_name = 22;
string create_user_id = 23;
message AttributeType {
string code = 1;
string description = 2;
}
message IncidentEquipment {
string equipment_number = 1;
string equipment_type = 2;
string equipment_prefix = 3;
}
message Driver {
int32 driver_id = 1;
string driver_first_name = 2;
string driver_last_name = 3;
}
}
from kloadgen.
@Rivarsal Thank you so much for this update. I have been pleased with the Avro portion of this extension and look forward to the Protobuf!
Great work to this team for the functionality that is available thus far.
from kloadgen.
HI @madidaye ,
just a small update. We are working on it and is possible to get something for you to test close to the end of the month.. may be a bit later, but not too much.
Cheers!!
@kszosze Thank you
from kloadgen.
Hello! @nrellanti
Just a quick update! We are almost done with the protobuff support, we had been delayed trying to investigate about external imports in the protobuf file. We still don´t have that feature developed, but we will release the first version of the protobufs around next week (this time for real) and we will try to find a way to develop support for external imports in the near future.
Sorry for all the troubles and delays, it has been a ride for sure! 😅
Cheers!
@Rivarsal Thank you so much for this info.
from kloadgen.
Hello @nrellanti @madidaye I just wanted to let you know that there is a pull request for the protobuf support. The number is #162 and the branch is called protobuff-support-new if you want to checkout to that branch.
You can try to download it and see if it works for you!
from kloadgen.
Hello @nrellanti @madidaye I just wanted to let you know that there is a pull request for the protobuf support. The number is #162 and the branch is called protobuff-support-new if you want to checkout to that branch.
You can try to download it and see if it works for you!
@Rivarsal , Thank you very much for the update, I will download and test for my script and let you know the output of the results.
from kloadgen.
Hi @nrellanti.
Had you tested your script?
Any feedback will be really appreciated. Keep in mind, we are still working to support imports from external proto files.
Cheers!
from kloadgen.
I am currently testing this protobuf functionality, and I am having trouble with the Value Schema Load Generator Config element.
When I try to add the element to my test plan, I get this error:
ERROR o.a.j.JMeter: Uncaught exception in thread Thread[AWT-EventQueue-0,6,main] java.lang.Error: java.lang.reflect.InvocationTargetException
Can someone assist me with this?
I am using a plugin built from the protobuff-support-new branch of the repo. I built the plugin with the latest changes as of yesterday afternoon.
from kloadgen.
Hi @madidaye,
sure no problem to help, can you provide the full error? And is possible jmx and proto?
Cheers.
P.D. we are close to complete support for import external proto definition, I'll keep you posted in the loop.
from kloadgen.
Sure, @kszosze
Here is the full error:
2022-01-28 08:36:44,794 ERROR o.a.j.JMeter: Uncaught exception in thread Thread[AWT-EventQueue-0,6,main] java.lang.Error: java.lang.reflect.InvocationTargetException at org.apache.jmeter.testbeans.gui.GenericTestBeanCustomizer.<init>(GenericTestBeanCustomizer.java:250) ~[ApacheJMeter_core.jar:5.4.1] at org.apache.jmeter.testbeans.gui.TestBeanGUI.init(TestBeanGUI.java:425) ~[ApacheJMeter_core.jar:5.4.1] at org.apache.jmeter.testbeans.gui.TestBeanGUI.configure(TestBeanGUI.java:301) ~[ApacheJMeter_core.jar:5.4.1] at org.apache.jmeter.gui.tree.JMeterTreeModel.addComponent(JMeterTreeModel.java:162) ~[ApacheJMeter_core.jar:5.4.1] at org.apache.jmeter.gui.action.AddToTree.doAction(AddToTree.java:69) ~[ApacheJMeter_core.jar:5.4.1] at org.apache.jmeter.gui.action.ActionRouter.performAction(ActionRouter.java:87) ~[ApacheJMeter_core.jar:5.4.1] at org.apache.jmeter.gui.action.ActionRouter.lambda$actionPerformed$0(ActionRouter.java:69) ~[ApacheJMeter_core.jar:5.4.1] at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313) ~[?:?] at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:770) ~[?:?] at java.awt.EventQueue$4.run(EventQueue.java:721) ~[?:?] at java.awt.EventQueue$4.run(EventQueue.java:715) ~[?:?] at java.security.AccessController.doPrivileged(Native Method) ~[?:?] at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) ~[?:?] at java.awt.EventQueue.dispatchEvent(EventQueue.java:740) ~[?:?] at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) [?:?] at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) [?:?] at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) [?:?] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) [?:?] at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) [?:?] at java.awt.EventDispatchThread.run(EventDispatchThread.java:90) [?:?]
I have attached the jmx as a text file: Proto Test Plan.txt
The proto I am using is simple, I'm trying to send messages to a localhost Kafka cluster (no security)
syntax = "proto3";
package com.mycorp.mynamespace;
message SampleRecord {
int32 int_id = 1;
double double_id = 2;
string string_name = 3;
}
from kloadgen.
Thanks, @madidaye,
I'll check it.
Cheers
from kloadgen.
Hi @madidaye,
I’m trying to reproduce the error, but I could not. Can you check you only have one version of the plugin installed? Sometimes 2 jars live in the same class path and create conflicts.
Cheers
from kloadgen.
Hi @madidaye,
any luck about that?
Cheers!
from kloadgen.
Hi @madidaye, @nrellanti,
we have 2 branches supporting protobuff
- protobuff-support-new : which only support plain proto definition
- protobuff-import-test-support: which support proto with external imports. Last one require you provide the jar with the proto files in the jmeter classpath (unless you were using the basic google ones like timestamp which are included already).
If you can, please test them in your use cases will improve our work.
Cheers
from kloadgen.
@jemacineiras, @kszosze , I have been successfully created jar file and tested with protobuff-support-new branch with schema Example-Variable-Substitution.jmx file with all proto files in the protobuff folder except providedTest.proto.
Error_1a: Getting error with providedTest.proto file in this branch (Attached logs in Error_1a.txt)
Error_1b: No response for processing other proto file(Attached logs in Error_1b.txt)
Error_2: Unable to save JMeter test plan for Value Schema file Load Generator Config changes (image Attached Error_2.xlsx)
Error_3: Getting error during new jmx file open . After execution with Value Schema file Load Generator Config changes. (image Attached in Error_2.xlsx)
Error 4: with Example-Test-Plan-2.jmx (Image and logs attached in Error_2.xlsx)
Error 5:with Example-Test-Plan.jmx (Image and logs attached in Error_2.xlsx)
Note: I will test with protobuff-import-test-support branch and let you know the output
from kloadgen.
Hi @nrellanti,
thanks for the testing and amazing feedback, I took a look, and I'm going to suggest move all the testing to the protobuff-import-test-support which I made some changes regarding data generation and other improvements.
Cheers
from kloadgen.
Hi @jemacineiras,
protobuff-import-test-support has been tested. This branch was working as expected for Kafka messages in the JMeter.
Thank you very much for all the updates.
Few issues:
Value Schema File Load Generator Config: Schema Type by default is "AVRO"
Unable to save Schema Type to "PROTOBUF"
Steps to generate error
Click to save test plan on JMeter
There are test items that have not been saved. Do you wish to save before clearing the test plan?
Yes/No/Cancel
Click Yes: not saving test plan
Click No: Test plan closes
Click Open and select any jmx file: getting below error
class org.apache.jmeter.control.gui.TestPlanGui cannot be cast to class org.apache.jmeter.testbeans.gui.TestBeanGUI (org.apache.jmeter.control.gui.TestPlanGui and org.apache.jmeter.testbeans.gui.TestBeanGUI are in unnamed module of loader org.apache.jmeter.DynamicClassLoader @47fd17e3)
from kloadgen.
Hi @jemacineiras ,
One of our proto file is getting below error, would you please suggest!
ERROR o.a.j.t.JMeterThread: Error while processing sampler: 'Java Request'.
java.lang.UnsupportedOperationException: This field is not of enum type. (abc.LocationUpdate.shipment)
at com.google.protobuf.Descriptors$FieldDescriptor.getEnumType(Descriptors.java:1247) ~[protobuff_v2.jar:1.1.8.4]
at net.coru.kloadgen.processor.ProtobufSchemaProcessor.processFieldValueMappingAsEnum(ProtobufSchemaProcessor.java:225) ~[protobuff_v2.jar:1.1.8.4]
at net.coru.kloadgen.processor.ProtobufSchemaProcessor.next(ProtobufSchemaProcessor.java:107) ~[protobuff_v2.jar:1.1.8.4]
at net.coru.kloadgen.loadgen.impl.ProtobufLoadGenerator.nextMessage(ProtobufLoadGenerator.java:52) ~[protobuff_v2.jar:1.1.8.4]
at net.coru.kloadgen.sampler.KafkaSchemaSampler.runTest(KafkaSchemaSampler.java:151) ~[protobuff_v2.jar:1.1.8.4]
at org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSampler.java:197) ~[ApacheJMeter_java.jar:5.4.3]
at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:638) ~[ApacheJMeter_core.jar:5.4.3]
at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:558) ~[ApacheJMeter_core.jar:5.4.3]
at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:489) [ApacheJMeter_core.jar:5.4.3]
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:256) [ApacheJMeter_core.jar:5.4.3]
at java.lang.Thread.run(Thread.java:834) [?:?]
from kloadgen.
HI @nrellanti,
I'll check the save issue and back to you asap.
About the "field is not an enum type" I'm going to need a proto file as example in order to replicate the error.
Cheers
from kloadgen.
Hi @nrellanti,
I made some changes in one visual component, so now you should save and recover jmx files.
Do you mind test it with your files?
Cheers
from kloadgen.
Hi @jemacineiras,
Save issue tested and verified on jmx files. It was working as expected. Thank you so much!
The "field is not an enum type" error : attached proto file for your reference.
proto_test_file.txt
from kloadgen.
Hi @nrellanti,
The problem with the file you provide, looks like we have an issue identifying the inner type CarrierIdentifier.
I'll take a look into it.
Cheers
from kloadgen.
HI @nrellanti,
I fix some things in the branch and added your proto as example. Do you mind testing it?
Afterwards, I'll remove your file.
Cheers
from kloadgen.
Hi @jemacineiras,
New code has been tested and verified, shared proto_test_file is working as expected without any errors. Thank you!
However, it will be reproducing the "save" error as I mentioned in previous comment ( #75 (comment) ).
from kloadgen.
Hi @nrellanti,
we are really happy protobuff support is working for you. Really, thanks for the feedback.
About the save issue, I uploaded a couple of changes, so it should be fixed by now.
Cheers
from kloadgen.
Hi @nrellanti,
we are really happy protobuff support is working for you. Really, thanks for the feedback.
About the save issue, I uploaded a couple of changes, so it should be fixed by now.
Cheers
Hi @jemacineiras, I'm getting java: error: release version 5 not supported.
from kloadgen.
HI @nrellanti,
that's weird, I’ll take a look. Which JVM are you using?
from kloadgen.
Hi @jemacineiras,
I have used three different versions
jdk1.8.0_101: java: diamond operator is not supported in -source 1.5
(use -source 7 or higher to enable diamond operator)
zulu11: java: error: release version 5 not supported.
zulu17.32.13: Cannot determine path to 'tools.jar' library for 17 (C:\Program Files (x86)\zulu17.32.13)
from kloadgen.
Hi @nrellanti,
we move to Java11 some months ago, since it reached the end of live. So is understandable it fails with jdk8.
I will try with zulu11 and see why it’s complaining. As far as I remember, this error came due to JVM versions, meaning use a Java 11 bytecode in a java 8 jvm.
About zulu 17, I think the tools.jar was removed in Java 14 or 15 from the standard deployment, so I think some manual configuration should be done in order to support it (like copying the tools.jar into a lib folder) or some code changes.
I’ll try to back to you asap with more information about.
Cheers
from kloadgen.
HI @nrellanti,
I’ve been testing the protobuff and couldn’t reproduce your problem with the version 5
also fix some issue with the Schema Registry integration.
Could you please give me some feedback about? How I can reproduce that error?
Cheers!
from kloadgen.
HI @nrellanti,
I’ve been testing the protobuff and couldn’t reproduce your problem with the
version 5
also fix some issue with the Schema Registry integration.Could you please give me some feedback about? How I can reproduce that error?
Cheers!
Hi @jemacineiras, I have been testing updated protobuff-import-test-support branch and### could not reproduce the problem.
from kloadgen.
Related Issues (20)
- [Issue]: Avro schemas with references HOT 6
- [Documentation Issue]: Postman collection to import HOT 1
- [Issue]: `sasl.jaas.config` property is not applied HOT 5
- [Issue]: Error when all fields are optional in AVRO
- [Issue]: Error with JMeter 5.6.2 in connection with plugin kloadgen 5.6.5 HOT 18
- [Issue]: Complete the funcionality of "Value Schema File Serializer" HOT 1
- [Issue]: Can't load an Avro schema serializer from register.
- [Issue]: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target HOT 4
- [Issue]: NullPointerException when starting a test with a Key Schema File Serializer Config HOT 1
- [Issue]: JSON value shema serializer exception HOT 1
- [Issue]: Simple Key Config serializer exception HOT 1
- [Issue]: Error while processing sampler: 'Kafka Broker'. java.lang.NullPointerException: Cannot invoke "String.toUpperCase()" because "registry" is null HOT 10
- [Issue]: Consumer does not show consumed messages in JMeter
- [Kloadgen]: I want to integrate Kloadgen with Google-PubSub
- [Issue]: kloadgen based on jmeter 5.6.3
- [Issue]: Unable to produce messages with bytes field in Protobuf schema using kloadgen plugin HOT 5
- [Issue]: sasl.jaas.config property is not applied HOT 4
- Starting a simple project with Simple Value Config and Simple Key Config HOT 5
- Error filtering class org.apache.commons.compress.harmony.pack200.Segment, it will be ignored java.lang.IncompatibleClassChangeError: class org.apache.commons.compress.harmony.pack200.Segment can not implement org.objectweb.asm.ClassVisitor, because it is not an interface (org.objectweb.asm.ClassVisitor is in unnamed module of loader org.apache.jmeter.DynamicClassLoader @65ab7765)[Issue]: HOT 6
- [Issue]: Parameter values are not taking properly HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kloadgen.