Comments (3)
Ok, interesting, I've been playing with a vectorized version of the existing from_dataframe() and this is what I've found:
-
My new code is much faster with big dataframes (above ~250 rows) and much slower with smaller dataframes.
-
Given that most messages will have a low number of rows (streaming data, not batches of data), I think the current code should stay.
-
I still like to share my vectorized version for future reference, what's the best way? @peter-quix
-
Whilst doing this exercise, I found two bugs that need solving:
from quix-streams.
Btw, find here both to_dataframe() new and old and how they compare: https://github.com/quixio/quix-streams-to_dataframe_tests/blob/main/src/PythonClient/tests/quixstreams/manual/from_dataframe.ipynb
from quix-streams.
No longer relevant, new python code is not working the same way
from quix-streams.
Related Issues (20)
- Add publish for individual timestamp.
- Add scalar state type
- Add rolling window state helper
- Remove value typed properties on TimeseriesDataTimestamp object HOT 1
- State handling improvements
- Use .net LTS for C# and only use .net 8 for python builds HOT 2
- Merge events endpoint with Timeseries endpoint HOT 1
- Exactly once processing semantic
- Python 3.6 support
- Add ability to optionally overwrite values for existing timestamps when adding to the buffer HOT 2
- Consistent validation of parameter_id is missing at some places HOT 1
- Use client.id to connect to Confluent Cloud Kafka
- Raw Topic Producer to guarantee last message sent HOT 1
- Quix 0.5.6 App.Run in python doesn't automatically subscribe to topics
- Add more docs to windowing HOT 3
- How to calculate mean etc over multiple columns HOT 2
- SDK breaks when handling in array HOT 1
- 2.3.1 typings HOT 2
- Support [GZIP] Prefix in codec id for legacy quix packages
- Better documentation of MessageContext HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from quix-streams.