Comments (2)
Hello,
This not a bug. In your example the storage has no time to clean up old data.
The point of time when data gets deleted depends on several factors, the most important ones are:
- The Java GC
- The storage GC
- The storage GC configuration
- The storage internal cache
- The write load.
In your example you create a heavy write load that requires some tweaking of the storage housekeeping.
The Example below should perform better regarding the cleanup.
It ensures that the Java GC and storage GC are executed and sets a very small object live time for the storage cache and increases the time budget for housekeeping.
final EmbeddedStorageManager storageManager = EmbeddedStorage
.start(Storage.ConfigurationBuilder()
.setEntityCacheEvaluator(Storage.EntityCacheEvaluator(1000, 10))
.setHousekeepingController(Storage.HousekeepingController(100, 1000_000_000))
.setStorageFileProvider(Storage.FileProvider(storagePath)).createConfiguration());
for (int i = 0; i < 50; i++) {
storageManager.setRoot(new byte[1000000]); // 1 MB
storageManager.storeRoot();
}
System.gc();
storageManager.issueFullGarbageCollection();
storageManager.issueFullFileCheck();
from microstream.
Okay, thank you. I thought that explicitly calling housekeeping would delete unreachable data (as mentioned in #179), but in fact it is not guaranteed, and cannot be enforced, right?
Your snippet with "aggressive housekeeping" also gives me the same result (storage not shrinked).
from microstream.
Related Issues (20)
- How to serialize Throwable? HOT 1
- java.lang.ClassNotFoundException: sun.misc.Unsafe HOT 3
- Add InputStream/OutputStream option to Serializer
- Java 21 virtual threads issue HOT 7
- Switch to Java 9+ System.Logging instead of SLF4J HOT 2
- pom.xml - testing dependencies definition HOT 1
- javafx saving observabel list resulting in "Type not persistable: "class java.util.TimerThread" HOT 8
- Inconsistent timestamp of file HOT 1
- Storage transactions time stamp not monotone increasing
- Rest Interface Issue Using Quarkus HOT 2
- Create different configuration profiles for different use cases
- Building the MicroStreamTransactionFileConverte HOT 1
- TransactionFileConverter fails to create output file
- microstream reports one.microstream.storage.exceptions.StorageExceptionConsistency: No entity found for objectId 1000000000001447523 HOT 1
- StorageExceptionDisruption Error HOT 9
- Inconsistent element count commig after store call HOT 2
- Improve StorageExceptionDisruptingExceptions
- why the microstream write performance is not good as DB? microsteam write cost is about 100ms+ HOT 1
- Cache get(...) returns null w/ expired entry and read-through enabled
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from microstream.