Comments (4)
@spenczar I'm not sure that's referring to invalid records – I think that's referring to records that fail to be written because of, for example, throughput limits being reached, or problems with the kinesis backend – hence why it talks about retrying them.
If you've seen otherwise can you give me some examples of the sorts of records you're talking about?
from kinesalite.
@spenczar how'd you go looking into this? Can you give me some example PutRecords calls where Kinesalite doesn't match Kinesis' behaviour?
from kinesalite.
You're correct, and I was mistaken here, @mhart - I had observed some records get rejected for being oversized by Kinesis, and misinterpreted my own logs. Throughput limits and internal errors are the only way I can see this happening. Thanks for clearing up my confusion.
from kinesalite.
No probs! In an ideal world I'd incorporate ways in here to simulate throughput limits etc, but that's kind of a bigger task than I've got time for I'm afraid.
from kinesalite.
Related Issues (20)
- SerializationException thrown when CBOR is enabled in java client
- Invalid base64
- Kinesalite not allowing colon character in tags, though AWS does allow
- Json Unmarshal Exception with AWS java SDK 2.X HOT 3
- Kinesalite allows splitting the same shard multiple times
- kinesis object read from stream have different capitalization than Kinesis
- Does not work with @aws-sdk/client-kinesis (v3) HOT 3
- Reading from kinesalite stream works, but writing to it fails with security token invalid error message HOT 4
- Inaccessible host localhost error when calling the createStream method and using NodeJS 16
- Failed to disable certificate validation, KCL with Kinesalite ssl enabled
- Logstash kinesis-input-plugin with kinesis lite?
- kinesis get-shard-iterator --shard-iterator-type AT_TIMESTAMP not return response HOT 3
- npm audit security report level-sublevel HOT 1
- Amount of shards are always 0 HOT 2
- High cpu
- Error when GetShardIterator by timestamp
- UnknownOperationException thrown by Java AWS SDK during ShardSyncTask HOT 1
- server-crt.pem file is expired
- Creating streams takes rather long HOT 2
- Implement NextToken/MaxRecords for ListShards
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kinesalite.