[Discussion] Test Scenario Runners

[bigs]

[Discussion] Test Scenario Runners

[bigs]

@anacrolix yeah, this is absolutely lovely. i'm definitely a YAML fan, and it's a drop in replacement so i'm totally okay with that. as far as the "steps" go, i don't want to recreate docker, so i'd want to limit the instruction set to a well specified DSL. that's what i'm interested in establishing.

my gut feeling is that for sake of learning what we want to do and getting something done quickly, we start by just exposing a programmatic API for controlling daemons via the control protocol. this API will include:

• environment variables for consul access
• command line options/environment variables detailing which "segment" of the network they are to control (this will correlate to a service in consul)
• seamless api for leveraging these inputs to create daemon clients
• simple api for performing basic tasks over a collection of daemon clients
  • churn (kill and redeploy task group)
  • periodic DHT queries (fixed / random CIDs)
  • peer discovery

[bigs]

thanks! fortunately, we don’t need it to execute what we want.

…

On Tue, Mar 12, 2019 at 03:13 Adrian Lanzafame ***@***.***> wrote: @yusefnapora <https://github.com/yusefnapora> @bigs <https://github.com/bigs> just making sure you are aware the service segmentation is an enterprise feature of Consul and not a part of the open source version. Not saying it isn't worth paying for, just that we will have to pay. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AANBWileCOkK-lM27GKftSv4DXwOpMrXks5vV1ObgaJpZM4a0yky> .

[yusefnapora]

Could we use Consul's service segmentation to simulate network partitions? It would be nice when testing e.g. circuit relay to know that two nodes can't communicate directly and have that enforced by the test setup.

[anacrolix]

YAML might be better suited, particularly for embedding snippets of stuff for other languages (bash etc.), keys, parts of scenarios etc., and the fact that humans will be writing these?

As a potential writer of scenarios I'd expect to be able to describe:

global_config:
  # some stuff like run name, permissions, consul and coordinator service configurations etc.
nodes:
  - num: 50
     network: &nat # some nat nonsense
     steps:
       - &install |
         go get dht
         some_other_install
       - |
         generate_data
         run_dht
     metrics: # how to generate metrics
  - num: 500
    <<: *nat # run the same nat as the above guys
    steps:
      - *install
      - |
        run_dht_to_get_data
    metrics:

Probably some synchronization will be necessary so nodes are in expected states before certain steps run. Some variance between nodes might be necessary, for example exposing an instance ID as an environment variable or something so instances can deterministically generate random data/keys/peer IDs etc.

Is this the kind of input you're after?

[bigs]

@yusefnapora interesting idea! partitions are not yet on the roadmap, but would love some more ideas on that matter!

[lanzafame]

@yusefnapora @bigs just making sure you are aware the service segmentation is an enterprise feature of Consul and not a part of the open source version. Not saying it isn't worth paying for, just that we will have to pay.