Comments (8)
Yes, the scripts mentioned above come from the old Netflix ChaosMonkey tools. They have now re-written that framework in Go and from what I can tell it's targeted at commercial cloud infrastructure. I'm not sure how much work it would be to fork it and add our stuff to it. Your question led me to look again at alternatives and I found https://harness.io/blog/chaos-engineering-tools/ which references several possibilities in a new discipline called Chaos Engineering (which sounds way cooler than Agitation by the way)
from accumulo-testing.
This sounds like it could be a whole project on its own, rather than something specifically for testing Accumulo. I'd want to be careful about scope creep here. Perhaps if that project were created separately, then maybe we could just add config/plugins to make use of it for testing Accumulo?
My suggestion was merely to redo the packaging for the existing agitator functions (stop/start DN, NN, TS,etc) and maybe to add new functions to introduce host issues. I was thinking something simple as my example above.
from accumulo-testing.
So, one idea I had was to make the agitator more dynamic, and make it so that you only had to run one script instead of multiple. You could, as an example, have a script that runs commands randomly from files in a directory. I tested this out, but it looks like my test files have been wiped. The basic gist was to have a set of bash files in a directory, where each file had two functions do() and undo(). The main agitator script would randomly select a file from the directory, source it, call do(), wait for some period of time and then call undo().
from accumulo-testing.
This sounds like it could be a whole project on its own, rather than something specifically for testing Accumulo. I'd want to be careful about scope creep here. Perhaps if that project were created separately, then maybe we could just add config/plugins to make use of it for testing Accumulo?
from accumulo-testing.
I was able to reproduce my test pretty easily. Two script files and the driver.
#! /bin/bash
function do_it {
echo "do A"
}
function undo_it {
echo "undo A"
}
#! /bin/bash
function do_it {
echo "do B"
}
function undo_it {
echo "undo B"
}
#! /bin/bash
for file in $(find scripts -type f | shuf)
do
source "$file"
do_it
sleep 1
undo_it
done
from accumulo-testing.
It seems to me like there are two separate ideas here:
-
use Linux host commands to introduce various things into the environment - high CPU / IO usage, packet loss, packet latency, packet corruption, bad routing, etc.
-
redo the packaging for the existing agitator functions
Not sure if both of these should be handled together or on their own.
from accumulo-testing.
Actually, both came from me, and the idea is that 2 allows for 1 to be easily added.
from accumulo-testing.
Have you looked into existing open source utilities out there to provide agitation that we wouldn't have to maintain, but can just use off the shelf to provide those extra features?
from accumulo-testing.
Related Issues (20)
- CopyTable thread died during Randomwalk MultiTable test run HOT 1
- RejectedExecutionException: Task org.apache.accumulo.core.trace.TraceWrappedRunnable HOT 3
- IllegalArgumentException: Rwalk Simpl.xml and Basic.xml module HOT 6
- Bad formatting when logging state information
- Security.xml throws exception with randomwalk HOT 7
- Logging is broken
- Terraform setup fails when using CentOS Stream 9 HOT 3
- Add pssh to packages to be installed via terraform
- agitator fails on CentOS 7.9 HOT 4
- Make bulk import random walk verify step print missing markers HOT 11
- Investigate pulling OpenTelemetry and Micrometer version from Accumulo repo HOT 1
- Agitator not randomly starting / stopping servers HOT 6
- Build stopped by files with unapproved licenses HOT 3
- Terraform setup fails with invalid function argument HOT 1
- Terraform setup fails due to old maven version HOT 2
- Create a split scaling test HOT 1
- Deprecated properties stopping the build
- ThriftPropertyException thrown on `cingest createtable` HOT 3
- mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
- Continuous ingest verify failing HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from accumulo-testing.