archipelproject / archipel Goto Github PK

View Code? Open in Web Editor NEW

800.0 75.0 126.0 44.17 MB

XMPP Based Orchestrator

Home Page: http://archipelproject.org

License: GNU Affero General Public License v3.0

Python 37.03% Shell 0.31% CSS 0.52% JavaScript 2.70% Objective-J 57.42% Ruby 0.14% HTML 1.89%

archipel's Introduction

What is Archipel?

Archipel is an Open Source project that aims to bring push notifications to virtualization orchestration using XMPP.

Archipel relies on open source technologies like python, ejabberd, cappuccino, strophe, nginx, libvirt, KVM and even more. All of these technologies are mixed together to create a real time orchestration solution for virtualization.

Thanks to Archipel you are be able to:

Visualize in real time the states of virtual machines and hypervisors
Send real time commands to them
Communicate inside the orchestrator with other human users
Organize virtual machines and hypervisors into groups and send commands to them
Connect distant datacenters in a secure way using XMPP S2S communication
Even use Archipel with XMPP client in order to receive your VMs notifications
Avoid the presence of SPOF. If a something fails, everything else still works
Use it with KVM, Xen, OpenVZ, VirtualBox, and mix them all if you want to
Have a great real time interface
And so much more to come!

Get help

Website : http://archipelproject.org
Nighty Builds : http://nightlies.archipelproject.org
Test application : http://app.archipelproject.org
Mailing list : http://groups.google.com/group/archipelproject
IRC : irc://irc.freenode.net/#archipel
Sources : https://github.com/archipelproject/archipel
Wiki : https://github.com/archipelproject/archipel/wiki
Bugtracker : https://github.com/archipelproject/archipel/issues
FAQ : https://github.com/archipelproject/Archipel/wiki/Faq

Installation

The complete installation procedure can be found at https://github.com/archipelproject/archipel/wiki. If you find some errors, please feel free to correct them.

Team

Antoine Mercadal (primalmotion) : Lead developer
Cyril Peponnet (CyrilPeponnet) : contributor
Nicolas Ochem (nicolasochem) : contributor
Franck Villaume (nerville) : contributor
Alexandre Fernandez (crashtest) : contributor
Ben Langfeld (benlangfedl) : StropheCappuccino contributor
David Weber (david-weber) : contributor
Anonymous (npmap) : contributor

Thanks

Alexandre Fernadez (crashtext) : main website hosting

License

Archipel is distributed under AGPL v3. See the LICENSE file

archipel's People

Contributors

Stargazers

Watchers

Forkers

xeonline snowlprd david-weber nadley flazcano basicer bassbauer nicolasochem dreamonxu wavesoft billthebest remotesyssupport ew0x trman lezval dhming double-z zzondlo rietn alibam picur everplays zouuup gbraad soulhunter cyrilpeponnet michelios amorican omshivaprakash myaime casualuser rnavarro someapp hj91 ahmedelshafaie lenineto nowaker canarytek spclops contegix blacklightops thalidzhokov ballacky13 liluxdev schubduese42 aya webrulon cpc26 markandrewj gotomypc fr34k8 qarekhani eignatov winneryong dogild zenny lidaoyou vaibhavbond kanewinter shellayin romanbogachev phieber linearregression benediktharter kris905 ruthger nealscarffery kienhung puppetninja rmoorman analani yashodhank pombredanne sksundaram crudbug alexeydeyneko crackmonkey mba811 folkcode ilmostro ed1023 cloudxtreme cechu66 pojen-huang jjoc errikozz jnaulty gocheap lamlai mmcduffie1 rowhit x11remix avitron01 brobridge nunb gvsurenderreddy w796933 wrestrtdr mhrubel cgvarela

archipel's Issues

Feature Request: Add more commands to hypervisor bot.

Please add more commands to the hypervisor bot.

The GUI is great, but for administration in general logging in to do simple things can be tedious versus querying the bot.

Currently the only command available is localize which has nothing to do with the state of the VMs.

To start, for my needs I would like be the ability to start/stop domains (I'm using libvirt qemu-kvm), and see capabilities. free ram would also be nice. Along with vncdisplay. Also commands that show the state of the host as well cpu/ram usage. If this plans to be a full blown interface with the same commands as libvirt, albeit probably cant edit the xml would be great.

Thanks,
Bryan

Couldn't find index page for 'archipel-agent-virtualmachine-vnc'

Hello everyone, I got an issue installing archipel agent:
easy_install archipel-agent
the following error appears:

Searching for archipel-agent
Best match: archipel-agent 1.0.21beta
Processing archipel_agent-1.0.21beta-py2.4.egg
archipel-agent 1.0.21beta is already the active version in easy-install.pth
Installing runarchipel script to /usr/bin
Installing archipel-initinstall script to /usr/bin
Installing archipel-updatedomain script to /usr/bin
Installing archipel-rolesnode script to /usr/bin
Installing archipel-tagnode script to /usr/bin
Installing archipel-importvirtualmachine script to /usr/bin

Using /usr/lib/python2.4/site-packages/archipel_agent-1.0.21beta-py2.4.egg
Processing dependencies for archipel-agent
Searching for archipel-agent-virtualmachine-vnc>=1.0.1beta
Reading http://cheeseshop.python.org/pypi/archipel-agent-virtualmachine-vnc/
Couldn't find index page for 'archipel-agent-virtualmachine-vnc' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading http://cheeseshop.python.org/pypi/
No local packages or download links found for archipel-agent-virtualmachine-vnc>=1.0.1beta
error: Could not find suitable distribution for Requirement.parse('archipel-agent-virtualmachine-vnc>=1.0.1beta')
[root@centossrv1 Archipel]# easy_install archipel-agent
Searching for archipel-agent
Best match: archipel-agent 1.0.21beta
Processing archipel_agent-1.0.21beta-py2.4.egg
archipel-agent 1.0.21beta is already the active version in easy-install.pth
Installing runarchipel script to /usr/bin
Installing archipel-initinstall script to /usr/bin
Installing archipel-updatedomain script to /usr/bin
Installing archipel-rolesnode script to /usr/bin
Installing archipel-tagnode script to /usr/bin
Installing archipel-importvirtualmachine script to /usr/bin

Dead outlet

There is a dead tabViewDevicesContainer outlet in VirtualMachineDefinitionController

Shared groups don't allow users to see the members as online

I tried to add a 'hypervisors' group and add my hypervisors to it for all users to see. Unless the users were also part of the hypervisor group they couldn't see the hypervisors (intentionally not in their roster). Of course, the issue then is that users are also in the hypervisor group, not what I had intended. Reading through the shared rosters documentation (http://www.process-one.net/docs/ejabberd/guide_en.html#htoc60), it appears that I need to have the users in a users group and put the hypervisors in it's own group. Then add the hypervisors to the users displayed group. Doing this from the ejabberd admin, let me see the hypervisor, but it was offline (probably because I never officially subscribed to it). Adding the users group to the hypervisor displayed group allowed me to make the hypervisor green - but archipel-agent fell into a connect/disconnect loop. At one point I did have this stable, but I added a vm and archipel-agent fell back into it's connect/disconnect loop

easy_install missing prerequisite description

here:
https://github.com/primalmotion/Archipel/wiki/Installation:-Agent
described the installation process and I just found that after invoking

easy_install archipel-agent

I got

Best match: SQLObject 0.16dev
Downloading http://svn.colorstudy.com/SQLObject/trunk#egg=SQLObject-0.16dev
Doing subversion checkout from http://svn.colorstudy.com/SQLObject/trunk to /tmp/easy_install-St0z8l/trunk
sh: svn: not found
Processing trunk
error: Couldn't find a setup script in /tmp/easy_install-St0z8l/trunk

so please add the subversion as necessary prerequisite

Missing symlink to TNKit

In /ArchipelClient/Frameworks/Debug missing TNKit link to ../../../TNKit/
GUI won't launch without this.

Creating shared group with spaces in description fails

DEBUG   ::2011-02-03 12:14:44::__init__.py:78::TNArchipelHypervisor.iq_group_create ([email protected]/xenophobia)::console command is : su - ejabberd -c '/opt/ejabberd-2.1.6/bin/ejabberdctl srg-create d3d3a
140-1752-5c21-48e7-4545c8408c3c xenophobia.ucsecurity.com "'Hypervisors'" "'All Hypervisors'" d3d3a140-1752-5c21-48e7-4545c8408c3c'
ERROR   ::2011-02-03 12:14:44::__init__.py:165::.iq_group_create: exception raised is : EJABBERDCTL command error : su - ejabberd -c '/opt/ejabberd-2.1.6/bin/e
jabberdctl srg-create d3d3a140-1752-5c21-48e7-4545c8408c3c xenophobia.ucsecurity.com "'Hypervisors'" "'All Hypervisors'" d3d3a140-1752-5c21-48e7-4545c8408c3c'

From the command line

 # su - ejabberd -c '/opt/ejabberd-2.1.6/bin/ejabberdctl srg-create d3d3a140-1752-5c21-48e7-4545c8408c3c xenophobia.ucsecurity.com "'Hypervisors'" "'All Hypervisors'" d3d3a140-1752-5c21-48e7-4545c8408c3c'
Hypervisors: 1: Syntax error: Unterminated quoted string

I believe the inner single quotes break the command, however removing them is also incorrect as "All Hypervisors" gets passed as two arguments.

It looks like ejabberd definition of string differs from mine

ejabberd@xenophobia:~$ /opt/ejabberd-2.1.6/bin/ejabberdctl srg-create d3d3a140-1752-5c21-48e7-4545c8408c3c xenophobia.ucsecurity.com 'Hypervisors' 'All Hypervisors' d3d3a140-1752-5c21-48e7-4545c4545c8408c3c

Error: the command "srg-create" requires 1 less argument.

Typo bootstrap/LPKit

Hi,
I have used git to download source, when launching bootstrap (which dl submodules) there is an error during LPkit build.

In bootstrap line 367 there is : jake -f myJakefile build , but in LPKit file is called myJakeFile.

It's boring because each time there is an error in bootstrap i have to answer all question again...

automatic creation of /vm sub-directories

would be really nice to have bootstrap automatically create directory that doesn't exist yet such as /vm/iso

Missing Scheduler lib?

Hi,
I have this error when i'm trying to start archipel :

$Starting runarchipel.py: /usr/local/lib/python2.6/dist-packages/xmpppy-0.5.0rc1-py2.6.egg/xmpp/auth.py:24: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
import sha,base64,random,dispatcher,re
/usr/local/lib/python2.6/dist-packages/xmpppy-0.5.0rc1-py2.6.egg/xmpp/auth.py:26: DeprecationWarning: the md5 module is deprecated; use hashlib instead
import md5
Traceback (most recent call last):
File "/usr/bin/ArchipelServer/runarchipel.py", line 103, in
load_modules()
File "/usr/bin/ArchipelServer/runarchipel.py", line 54, in load_modules
import(module_dir + "." + module, None, locals())
File "/usr/bin/ArchipelServer/modules/actions_scheduler/init.py", line 2, in
import actionscheduler
File "/usr/bin/ArchipelServer/modules/actions_scheduler/actionscheduler.py", line 25, in
from apscheduler.scheduler import Scheduler
ImportError: No module named apscheduler.scheduler

where can i find this?

There is some problems with bootstrap on Debian, as soon as install succeed i'm going to write needed change.

OOM killer protection

Add a checked button to say :"Please do not kill this vm if you run out of memory"

Template configuration file for Archipel

Use the option values of bootstrap in the configuration file of Archipel

Set user rights for exec files and directories in bootstrap

Native Host MainMenu no label

Main menu items are present, but they have no label

LDAP authentication

It would be nice to see LDAP authentication available. Since most (all) of the server side code is in python there is a Python Module available, "python-ldap."

HypevisorNetwork doesn't display any network

HypevisorNetwork doesn't display network during reloading.

Consolidate sqlite3 files

as number of sqlite databases grow, it would be cool to use one or two global db connection and implement a real model. Performances will be better.

Remove useless datasources in HealthModule

The use of different identical class is useless.

missing scrollbar in definition tab

using chromium 11.0.657.0 (73275)

select a VM, resize the browser.... no horizontal scrollbar.

Add support for resizing

Ubuntu compatible

It may be a dumb question but does/will archipel compile/run on Ubuntu ?

From now i get a big and fat ' ERROR: You need easy_install'

[Client] No automatic connection

It's in the title : no automatic connection in the client. Let the user press the "Connect" button, instead of having to reload the client each time a connection parameter changes.

installation fails on apscheduler

........
Installed /usr/local/lib/python2.6/dist-packages/archipel_agent-1.0.8beta-py2.6.egg
Processing dependencies for archipel-agent==1.0.8beta
Searching for apscheduler>=1.3.1
Reading http://pypi.python.org/simple/apscheduler/
No local packages or download links found for apscheduler>=1.3.1
Best match: None
Traceback (most recent call last):
File "setup.py", line 89, in
('system/etc/archipel/' , ['install/etc/archipel/archipel.conf', 'install/etc/archipel/vnc.pem'])
File "/usr/lib/python2.6/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib/python2.6/distutils/dist.py", line 975, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.6/distutils/dist.py", line 995, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/install.py", line 76, in run
File "/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/install.py", line 104, in do_egg_install
File "/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 211, in run

File "/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 427, in easy_install

File "/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 478, in install_item

File "/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 519, in process_distribution

File "/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 563, in resolve
plugin_env, full_env=None, installer=None, fallback=True
File "/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 799, in best_match

File "/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 811, in obtain
self.add(other)
File "/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py", line 434, in easy_install

File "/usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg/setuptools/package_index.py", line 475, in fetch_distribution
AttributeError: 'NoneType' object has no attribute 'clone'

arcihpel-agent installation error

hello everyone, I gott the following installing archipel agent:
easy_install archipel-agent

ArchipelClient: problem running bootstrap

Performed the Agent installation, then followed instruction for Client (on the same machine).
Here is the output:

Archipel Client

Welcome to the installation script of Archipel Client.
Note that you must be root in order to install this software. Please answer the
following questions.

Testing environment

Would you like to build Cappuccino ? [y/n] : y
Would you like to build GrowlCappuccino ? [y/n] :
Would you like to build VNCCappuccino ? [y/n] :
Would you like to build StropheCappuccino ? [y/n] :
Would you like to build iTunesTabView ? [y/n] :
Would you like to build MessageBoard ? [y/n] :
Would you like to build TNKit ? [y/n] :
Would you like to build LPKit ? [y/n] :
Would you like to build Archipel Client ? [y/n] :
Would you like to apply Cappuccino Framework ? [y/n] :
Would you like to export Archipel ? [y/n] :
Where do you want to export Archipel Client ? [~/] :

Installation information
- build Cappuccino : True
  - build GrowlCappuccino : True
  - build VNCCappuccino : True
  - build StropheCappuccino : True
  - build iTunesTabView : True
  - build MessageBoard : True
  - build TNKit : True
  - build LPKit : True
  - build Archipel Client : True
  - apply Cappuccino : True
  - export to : ~/
Do you confirm ? [y/n] : y
Building Cappuccino

Narwhal JavaScript platform is required. Install it automatically now?
Enter "yes" or "no":

yes

To use the default location, "/usr/local/narwhal", just hit enter/return, or enter another path:

Downloading Narwhal from "http://github.com/280north/narwhal/zipball/master"...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 4062 100 4062 0 0 8410 0 --:--:-- --:--:-- --:--:-- 3966k
Installing Narwhal...
Archive: /tmp/narwhal.zip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of /tmp/narwhal.zip or
/tmp/narwhal.zip.zip, and cannot find /tmp/narwhal.zipArchopelClient: problem running bootstrap.ZIP, period.
Error: problem running boostrap.sh. Exiting.
Unable to bootstrap Cappuccino

ubuntu : /var/lock/subsys/archipel

On ubuntu that directory doesn't exist; should be '/var/lock/archipel' instead.

archipel-importvirtualmachine restart required

if I am going to add a already existing Virtual Machine with 'archipel-importvirtualmachine' I cannot see this machine live in the client.
It does not work, also, if I log out and log in from/in the Archipel Client.
I have to restart Archipel Agent to add existing VMs.

Adding "Cancel" button

I think it will be cool to have a "Cancel" button on each popup window, because the close button of the window it is very far.

Another idea is to had a binding with the "Esc" key to close the window

Ubuntu/Debian packaging

Hi,

The project looks good but quite non-trivial to deploy. Do you guys have the plans to simplify installation process by creating packages for several well known distributions in the nearest future?

Thanx.

Add CPU and Hypervisor type to default machine definition

Virtual Machine preferences window has just about every default except CPU and Hypervisor. A preferred default network (and NIC type) would be nice also.

remove all useless global scope variables

There's a lot of cases where global variables should be just file-scoped, not app-scoped.

missing scrollbar in xml editor

in vm deifnition, manual xml editor is unusable. vertical scrollbar is missing....

ubuntu : starting server

/etc/init.d/archipel start

Traceback (most recent call last):
File "/usr/bin/ArchipelServer/runarchipel.py", line 115, in <module> main()
File "/usr/bin/ArchipelServer/runarchipel.py", line 72, in main
hyp = archipel.TNArchipelHypervisor(jid, password, config, name, database)
File "/usr/bin/ArchipelServer/archipelHypervisor.py", line 122, in __init__
names_file = open(self.configuration.get("HYPERVISOR",      
"name_generation_file"), 'r')
IOError: [Errno 2] No such file or directory: 'names.txt'

Error occurs while trying to create new VMs

When I create new VMs, I get Errors and those does not want to subscribe themselves.

Here my logfile, archipel.conf and ejabberd.cfg
https://gist.github.com/819408

Use real VM names in libvirt <name> tag instead of UUID

The UUID tag (as well as description) already have the UUID in it and the output of virt-top (http://people.redhat.com/~rjones/virt-top/) as shown below, would be much more useful and allow admins to easily understand what the VMs are doing when operating from the command line.

virt-top 18:18:19 - x86_64 6/6CPU 3000MHz 16080MB
8 domains, 7 active, 7 running, 0 sleeping, 0 paused, 1 inactive D:0 O:0 X:0
CPU: 4.7%  Mem: 3968 MB (3968 MB by guests)

   ID S RDRQ WRRQ RXBY TXBY %CPU %MEM    TIME   NAME                                                                                                 
   17 R    0    0  210    0  0.8 12.0  92:25.80 4ddb84ec-3594-11e0-a2ed-1c6f653df42a
   19 R    0    0   52    0  0.7  3.0  49:55.54 81715502-3594-11e0-a2ed-1c6f653df42a
   14 R    0    0  210   46  0.7  2.0  55:24.36 e870aec0-34c6-11e0-a2ed-1c6f653df42a
   10 R    0    0  210    0  0.6  1.0 197:51.53 3e7c849c-3594-11e0-a2ed-1c6f653df42a
   16 R    0    0  342    0  0.6  1.0  45:36.33 59ee9efe-3594-11e0-a2ed-1c6f653df42a
   11 R    0    0  210    0  0.6  1.0 192:44.17 6321f89a-3594-11e0-a2ed-1c6f653df42a
   18 R    0    0  210    0  0.6  1.0  42:37.76 7076ec6c-3594-11e0-a2ed-1c6f653df42a
    -                                           (00cb193c-3027-11e0-a2ed-1c6f653df42a)

random erreur about VirtualMachineDefinition

Sometimes, when hypervisor stanza definition is delayed, Archipel crashes

Use capp_lint on Archipel Core

We need to capp lint a little.

ubuntu: daemon config

$Starting runarchipel.py: daemon: fatal: syntax error in /etc/archipel/archipel.conf, line 1:
[LOGGING]

Python version check

We need to add a check version of python.

GUI : not found

I was able to successfully install client; launched it but when browsing :
http://192.168.1.37:8080/

I get :
Not Found: /

can't login into gui

I've been finally been able to setup the client.

created a user using
ejabberdctl register test localhost test

On GUI i have
jabber-id : test@localhost
bosh-service : http://192.168.1.37:5280/http-bind

and get an 'internal-server-error'

ArchipelClient fail to load file TNStropheIMClient.j

With the current master branch I can only see the spinner, running forever. In the browser log I see the following error: "Uncaught Error: Could not load file StropheCappuccino/TNStropheIMClient.j" at File Objective-J.js Line 3227
The problem doesn't appear at the last nightly build.
I tried to fix this by myself but I have no idea where to start searching.

Btw. there a few Info.plist files which are at the wrong directory and fail to load. Also at the last nightly. But it seems they don't cause any problems. Only ugly logs ;)

Health statistics for virtual machines [renamed by primalmotion]

While libvirt doesn't appear to support this directly, it does provide CPU time used and as can be seen with (virt-top http://people.redhat.com/~rjones/virt-top/) it's possible to use this information to create usage deltas and get a cpu usage percentage.

Fix native host copy/cut/paste issue

It is not possible to use copy / cut / paste on Native Host

Race condition when starting VM to get the VNC server address

sometimes we ask for vnc server before it's really started

FusionForge plugin

the main idea is to offer from fusionforge the ability to deploy code into VM. Probably need Continuus Integration plugin too.

need real analysis on how to do it.

VNC focus never released

Sometimes, when using VNC Screen (from external window or embeded one) the focus is never given back to the application when window is closing. This then need to be reloaded

Typo in /etc/init.d/archipel

if [[ which daemon != "" ]]; then
daemonCommand=deamon
fi

remove : [[ ]] and deamon should be daemon

VirtualMachineDefinition module Refresh Device TableView problem

The Device's TableView doesn't reload correctly le TableView before resizing the window.

Chrome can't load files

After a build then copy of the build release in /var/www/Archipel, chrome displays these errors in console :

Failed to load resource: the server responded with a status of 404 (Not Found)
/Archipel/Frameworks/TNKit/TNKit.j
Failed to load resource: the server responded with a status of 404 (Not Found)
/Archipel/Frameworks/Debug/TNKit/Info.plist
Failed to load resource: the server responded with a status of 404 (Not Found)
/Archipel/Frameworks/StropheCappuccino/Resources/Strophe/Info.plist
Failed to load resource: the server responded with a status of 404 (Not Found)
/Archipel/Frameworks/VNCCappuccino/Resources/Info.plist
Failed to load resource: the server responded with a status of 404 (Not Found)
/Archipel/Frameworks/Debug/TNKit/TNKit.j
Failed to load resource: the server responded with a status of 404 (Not Found)
/Archipel/Frameworks/Objective-J/Objective-J.js:3223
Uncaught Error: Could not load file at TNKit/TNKit.j

files (TNKit Missing) aren't in the : /ArchipelClient/Build/Release/Archipel/Frameworks/

Native app generation doesn't copy modules

Generated App is module empty