stackstorm / packer-st2 Goto Github PK
View Code? Open in Web Editor NEWPacker templates for building & deploying Vagrant & OVA with StackStorm installed
Home Page: https://app.vagrantup.com/stackstorm/boxes/st2
License: Apache License 2.0
Packer templates for building & deploying Vagrant & OVA with StackStorm installed
Home Page: https://app.vagrantup.com/stackstorm/boxes/st2
License: Apache License 2.0
Naturally it's always nicer to have the linting checks as a dedicated jobs in the CI pipeline like it was before (`inspec-lint`, `packer-lint` running in parallel as a dependency to the main "packer-build" task). Provides some better visuals and less monolytics.
But because there was already a lot of work, not a requirement. Just nice to have, if you'll have a few more cycles to spend on it.
Originally posted by @armab in #64 (comment)
Releasing/Deploying a Vagrant box should be done automatically within st2cd release workflows.
Once a new StackStorm version is released, - run automation to deploy new Vagrant box.
Automation sketch that workflow should do:
Related to https://github.com/StackStorm/ova/issues/104
The way we'll distribute OVA is via Github Releases.
On every new git tag (aka Github release) CircleCI will build and upload the artifact to respective Github release page.
It means we should also start versioning the OVA and add CHANGELOG.md
for this repo which is a good thing.
Include Vagrant & OVA Virtual Appliance as an installation method in st2docs.
Cover basic steps:
As part of the maintenance work, switch from Ubuntu Xenial
which relies on py2 to Ubuntu Bionic
with python3
and MongoDB 4 for the Vagrant box we produce.
Add to the Packer pipeline steps to deploy produced Vagrant boxes to Vagrant Cloud/Atlas.
Start with Virtualbox provider, but eventually should be done for VMware boxes as well.
There are concerns about security keys generated during the CI process and so considered insecure.
And while Vagrant is considered as a solution for demo purposes, OVA could be used in prod environments.
The following needs to be auto-generated on first VM boot:
Stanley
private SSH key & take care of Authorized keySolution would be systemd
configs which rely on ssh/nginx services (runs as "before" dependency) and creates keys/certs if they're not there already.
This will ensure that SSH/SSL secrets are unique and not the same across all the users after our Packer builds.
Apart of that #17, - need to document as insecure:
vagrant
Linux user with default vagrant
passwordvagrant
Linux user with default/known Hashicorp SSH private keyTo have fully operable VM running under Virtualbox, guest additions should be installed.
See https://www.packer.io/docs/builders/virtualbox-iso.html#guest-additions
During the deploy
CircleCI stage, parse CHANGELOG
and copy-paste list of changes into Github Releases page.
Apart of that, include:
vagrant init stackstorm/st2 --box-version ${CIRCLE_TAG}
vagrant up
When someone SSH-in into a running container EWC/Enterprise is mentioned.
Remove those references as EWC is opensourced now:
https://stackstorm.com/2020/05/27/extreme-networks-donates-ewc-to-linux-foundation/
Looks like a SIGTERM is received prior to running integration tests:
https://circleci.com/gh/StackStorm/packer-st2/79
This is an intermittent bug. Re-running CI again can "fix" the issue.
Per https://www.packer.io/docs/other/debugging.html#debugging-packer, use PACKER_LOG=1
to see if that sheds more light on what's going on...
Hello,
I am getting the following error, which I am unable to override.
The IP address configured for the host-only network is not within the
allowed ranges. Please update the address used to be within the allowed
ranges and run the command again.
Address: 10.10.10.10
Ranges: 192.168.56.0/21
Valid ranges can be modified in the /etc/vbox/networks.conf file. For
more information including valid format see:
https://www.virtualbox.org/manual/ch06.html#network_hostonly
From what I understand the need to be changed or ST2 needs to run on a different IP address.
The v3.7.0 build in the CircleCI CICD pipeline is failing with the below issue:
TASK [Create server] ***********************************************************
FAILED - RETRYING: Create server (5 retries left).
FAILED - RETRYING: Create server (4 retries left).
FAILED - RETRYING: Create server (3 retries left).
FAILED - RETRYING: Create server (2 retries left).
FAILED - RETRYING: Create server (1 retries left).
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: Error: Error 422: sjc1 is not a valid facility
fatal: [localhost]: FAILED! => {"attempts": 5, "changed": false, "msg": "failed to set device state active, error: Error 422: sjc1 is not a valid facility"}
to retry, use: --limit @/root/ova/.circleci/ansible/create.retry
PLAY RECAP *********************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=1
Exited with code exit status 2
CircleCI received exit code 2
Because of this failure, there is no updated Vagrant box in Vagrant cloud.
https://app.vagrantup.com/stackstorm/boxes/st2
At the moment we build ST2 vagrant box + OVA by requesting a bare-metal instance at Packet.net, making a Packer build there and deploying produced artifact to the Vagrant Cloud.
Turns out GH Actions can now support Vagrant + Virtualbox (via nested virtualization? ).
Explore/research the https://stackoverflow.com/a/60380518/4533625 and https://github.com/jonashackt/vagrant-github-actions and moving the pipeline to GH Actions which will save us some $ and complexity by not using external Packet.net bare-metal instance. Side effects could be: slow/unstable build process, needs evaluation.
Add the Hashicorp private key for vagrant
linux user to get the vagrant up
working.
The use of a /31 prefix for the host network does not work on all operating systems. See StackStorm/st2docs#744
This should be changed to /30.
Optimize ovf for VMware use with ovftool
& established processes from ova
repo.
Not sure yet if we need it for Community version.
We should make the following changes to the published image:
~/.st2/config
file which contains credentials is only readable by the owner (and also make sure that the owner user is correct, in vagrant case that should be vagrant
user)~/.st2/st2admin-token
file which contains cached admin auth token. It seems like that file is generated during the CI process and it's owned by root which means vagrant
user can't read / write to it. We should delete it before generating the final artifact.From Slack:
@kami yes, seen that before. It's indeed created during the CI runs.
It happens because we do `sudo st2 ...` somewhere and so token is saved with `root` permissions.
I even thought about submitting a PR for st2 so the dir `/home/${USERNAME}/.st2/` is created by StackStorm client with `setgid` flag, meaning token will take the folder's owner group.
But we were pretty time limited on this OVA work.
Can you create an issue in https://github.com/stackstorm/packer-st2 ?
Another issue I noticed during that work, - `.st2/config` has read-all permissions and so everyone can read st2 username:password.
This is bug by our `curl|bash` installer:
```$ ls -la ~/.st2/config
-rw-r--r-- 1 vagrant vagrant 54 May 23 14:09 /home/vagrant/.st2/config
During the st2cd
release automation, we need to include a message in CHANGELOG.md
when new st2 version is released.
Otherwise we'll be in a strange situation when new packer-st2
release/tag is present, - but CHANGELOG is empty.
Add MOTD/Welcome message with after logging in to default shell.
It'll show useful links to StackStorm docs, forums, Slack.
During the CircleCI deploy step (which happens on git tag/git release) we'll need to extract the ST2_VERSION
and BOX_VERSION
from the git tag.
That'll allow us to pin specific version, instead of using "latest" default.
One minor issue: we're trying to remove links to the forum whenever we can. Since your already editing this, could you adjust the links?
Otherwise, LGTM. It's wonderful to finally have this fixed!
Note: We will need to update other the release instructions and probably the st2cicd workflows for release after this gets merged.
Originally posted by @cognifloyd in #64 (review)
We should warn a user that insecure login credentials are used: vagrant:vagrant
and Hashicorp SSH key.
And so it's not recommended to use the box for prod-related environments before removing insecure creds.
Try to print a warning message via:
Add Inspec tests related to how system
is configured and works: Vagrant user accounts, etc.
Need to upgrade to 16.04.07 x64 iso
Open access to st2web UI via Virtualbox port fowarding.
Apparently, it's easy to do, having control over the Vagrant template we ship with the .box
.
Would be nice if the vagrant box could run in KVM for those of us running Linux.
I know on my host i have to stop KVM to get VirtualBox running.
Here's the snippet in my Vagrantfile that i edit after i clone st2vagrant
st2.vm.box = "elastic/ubuntu-16.04-x86_64"
st2.vm.provider :libvirt do |vb|
vb.host = "#{hostname}"
vb.memory = 2048
vb.cpus = 2
vb.uri = "qemu:///system"
vb.storage_pool_name = "images"
end
Add stackstorm
-related Inspec Tests
Link stackstorm tests with an executable entrypoint available in user's bin, so user can invoke StackStorm integration tests
Port Packet CI from ova
repo.
Currently the build is based on Ubuntu18 iso which will be dropped in st2 v3.9.0
Switch to U20 box, update https://github.com/StackStorm/packer-st2/blob/master/st2.json and other places.
We run full packer build pipeline in Github Actions.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.