inmanta/inmanta-core

Inmanta is an automation and orchestration tool

inmanta

Repository for the inmanta package. This package exists for backwards compatibility and depends on the inmanta-core package. For more information, have a look at inmanta-core's GitHub page.

inmanta-core's Issues

add command to create new module

Migrated from gitlab, originally by @wouterdb on Mar 16, 2016, 13:21

.gitignore
git
module.yml
directories
_init.cf
add to project

Visualize the model

Migrated from gitlab, originally by @wouterdb on Feb 22, 2016, 13:19

Export the model to create visualisations

Idea: annotate source in some way to support visualizing

Scheduling of Optional values is bad

Migrated from gitlab, originally by @wouterdb on Feb 19, 2016, 13:26

[0:1] relations should be scheduled as list or use of optionals should give an error

Now they are sometimes scheduled too early, giving bad results

Additionally, SetAttribute.setValue (as used by Constructor statement), when used with an optional as value does nothing, leaving an Unset (instead of optional). (Very hard to diagnose)

Navigating a [0:1] relationship in the model can be considered a syntactic error, as the user can not be sure that the instance on the other side will exist,...

Perhaps throwing an error would be best, as anny attempt to schedule would be tricky

solve parser warnings

Migrated from gitlab, originally by @wouterdb on Apr 19, 2016, 22:09

make the grammar unambiguous

Make method on Project to store the config

Migrated from gitlab, originally by @wouterdb on Mar 24, 2016, 16:06

Use resource serialisation framework instead of resource_to_id

Migrated from gitlab, originally by @bartv on Apr 18, 2016, 10:30

Selective push to agents

When a new configuration model is released, the agent pulls the entire configuration model for it is responsible for and deploys it. This feature should add a mode where only changes in the model are pushed to the agent for faster deployments. Every x minutes the agent than checks the changes for the entire configuration model.

Fix imports

Migrated from gitlab, originally by @bartv on Apr 15, 2016, 12:37

Imports should work for any module without having to include impera.app

for loops broken

Migrated from gitlab, originally by @bartv on Apr 24, 2016, 09:44

For loops are broken with the new compiler. This is required for the dnetcloud-config project. The bug can be reproduced with:

values = [1, 2, 3, 4, 5]

for i in values:

end

Add delete for environment

Migrated from gitlab, originally by @bartv on Dec 3, 2015, 10:17

make compiler test unit, to test evalution

Migrated from gitlab, originally by @wouterdb on Apr 22, 2016, 10:57

Records for forms are not consistent

Migrated from gitlab, originally by @wouterdb on Jan 18, 2016, 14:27

Sorting is fucked up
http://verdana.cs.kuleuven.be:8000/app/#/environment/0a25023f-de4d-4e26-ab39-a2f66897b98c/forms
Sort on subnet_id does not work at all.

see also: https://code.impera.io/platform/impera-dashboard/issues/26

Delete parameters of purged resources

Migrated from gitlab, originally by @bartv on Dec 3, 2015, 16:14

When a resources gets purged remove all known parameters.

Reorganize source code

Migrated from gitlab, originally by @bartv on Apr 25, 2016, 11:25

Reorganize source code in the following packages:
compiler
server
agent
common
plugins

strip stats module from compiler

Migrated from gitlab, originally by @wouterdb on Apr 22, 2016, 10:57

module validate command herstellen

Migrated from gitlab, originally by @wouterdb on Mar 16, 2016, 13:41

add command to commit module

Migrated from gitlab, originally by @wouterdb on Mar 16, 2016, 13:25

for dev/prod

Add concurrency to the server

Migrated from gitlab, originally by @bartv on Jan 7, 2016, 13:43

https://motorengine.readthedocs.org/en/latest/

Remove agents when env is removed

Migrated from gitlab, originally by @bartv on Dec 7, 2015, 13:26

Module yml parser crashes when dep does not contain a ,

Migrated from gitlab, originally by @bartv on Mar 11, 2016, 12:08

requires:
php: [email protected]:modules/php

This crashes the compiler.

Agents started by the server should have their own virtualenv and code

Migrated from gitlab, originally by @bartv on Mar 1, 2016, 14:24

Store the latest configuration on each agent for offline enforcement

Migrated from gitlab, originally by @bartv on Apr 27, 2016, 10:02

Make sure that the agent can also function without a server. The agent keep enforcing the configuration but without any new updates.

Remove parsetab.py from git

Migrated from gitlab, originally by @bartv on Apr 19, 2016, 21:17

Remove parsetab.py from git. Every little change in a file or ply install regenerates this script. The spec file for the RPM already regenerates this script. This should be added to the setup.py file so it is generic for every package system.

Write unit test to verify the correct implementation of indexes

Migrated from gitlab, originally by @bartv on Apr 20, 2016, 09:10

An index can be defined on attributes and relations with arity 1

Bug in instruction scheduling

Migrated from gitlab, originally by @wouterdb on Mar 31, 2016, 18:41

Ik heb een bug gevonden die me zeer verontrust.

neem volgende stack trace

Als je de exception negeert, en de flow volgt.
Frame per frame

1. scheduling gaat stmts sorteren
2. statement blijkt resolved te zijn, dus gaat zijn actions aan de graph hangen
   std::ConfigFile(host=master.host, content=bind::serial(domain, zone_content), path="/var/named/chroot/var/named/master/{{ domain }}.db", mode=644, owner="root", group="root", requires=dir, provides=r_sv)
3. call to build action list
4. build action list gaat kijken naar het type van de variable 'content' om te zien of het een list is, zodat de juiste actions gemaakt kunnen worden. Om het type te weten vraagt het de waarde op.
5. de waarde wordt aan de result variable gevraagd
6. de waarde wordt aan de lazy variable gevraagd
7. de functie bind::serial wordt aangeroepen (zonder ooit gescheduled te zijn?)

 File "/home/wouter/inc/impera/src/impera/execute/scheduler.py", line 295, in _sort_statements
    elif state.resolved():
  File "/home/wouter/inc/impera/src/impera/execute/state.py", line 235, in resolved
    self.graph.add_actions(self, self.statement.actions(self))
  File "/home/wouter/inc/impera/src/impera/ast/statements/generator.py", line 318, in actions
    instance_type=type_class.get_class_type()))
  File "/home/wouter/inc/impera/src/impera/ast/statements/assign.py", line 153, in build_action_list
    if isinstance(value_ref.value, list):
  File "/home/wouter/inc/impera/src/impera/ast/variables.py", line 362, in get_value
    self.__value_cache = result.value
  File "/home/wouter/inc/impera/src/impera/ast/variables.py", line 418, in get_value
    self.set_value(self._value_generator())
  File "/home/wouter/inc/impera/src/impera/ast/statements/call.py", line 111, in lazy
    result = function(*arguments)
  File "/home/wouter/inc/impera/src/impera/plugins/base.py", line 309, in __call__
    value = self.call(*new_args)
  File "/home/wouter/inc/impera/src/impera/plugins/base.py", line 352, in wrapper
    return fnc(*args)
  File "./libs/bind/plugins/__init__.py", line 57, in serial
    md5sum = hashlib.md5(zonefile.encode("utf-8")).hexdigest()

Op deze manier wordt de scheduler gewoon gebypassed??

Ik heb nog wat verder gezocht en het blijkt dat de resultvariable die zone_content bevat wel degelijk version = 1 heeft. Maar toch is de version None. Waar de result in de variable gestoken wordt heb ik nog niet uitgevist.

Wat ik wel al heb uitgevist is dat DynamicState.evaluate de enige plaats is waar de verions wordt opgehoogd. Daar wordt echter niet de value gezet. Een check in DynamicState.evaluate om te zien of de value gezet is is ook niet mogelijk, want er zijn statement (zoals assignment) die wel een result variable hebben, maar geen result terug geven.

To Be Continued

Modules from two sources

Migrated from gitlab, originally by @bartv on Apr 5, 2016, 14:41

Issues a warning when the same module is defined from two sources. Could happen with modules that are referenced a lot such as the std module.

Make module operations parrallel with asyncio

Migrated from gitlab, originally by @bartv on Apr 25, 2016, 11:45

Operations such as install, update and push are dead slow for large projects

Add vagrant file to support development

Migrated from gitlab, originally by @bartv on Apr 5, 2016, 13:47

Errors in deps from project.yml are printed wrong

Migrated from gitlab, originally by @bartv on Mar 11, 2016, 10:03

Module . requires module backuppc with version ">= 0.1" which is not loaded

. should be project or project.yml?

Unique key constraint on environment name is wrong

Migrated from gitlab, originally by @bartv on Apr 28, 2016, 21:44

This index forces each env to be unique and not per project

Move get_facts entirely to std module

Migrated from gitlab, originally by @bartv on Apr 25, 2016, 11:43

Complete rename from impera to inmanta

Migrated from gitlab, originally by @bartv on Apr 25, 2016, 11:19

Change remote url of env repo when url is changed through the API

Migrated from gitlab, originally by @bartv on Jan 3, 2016, 14:51

Add SSL to API transport

Migrated from gitlab, originally by @bartv on Apr 18, 2016, 21:38

Value is reported twice

Migrated from gitlab, originally by @bartv on Apr 25, 2016, 12:46

Traceback (most recent call last):
  File "/home/bart/.virtualenvs/impera/bin/impera", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/home/bart/workspace/impera/bin/impera", line 22, in <module>
    app()
  File "/home/bart/workspace/impera/src/impera/app.py", line 237, in app
    options.func(options)
  File "/home/bart/workspace/impera/src/impera/app.py", line 74, in compile_project
    result = do_compile()
  File "/home/bart/workspace/impera/src/impera/compiler.py", line 49, in do_compile
    success = sched.run(compiler, statements, blocks)
  File "/home/bart/workspace/impera/src/impera/execute/scheduler.py", line 188, in run
    next.execute()
  File "/home/bart/workspace/impera/src/impera/execute/runtime.py", line 359, in execute
    raise e
  File "/home/bart/workspace/impera/src/impera/execute/runtime.py", line 356, in execute
    self.result.set_value(value)
  File "/home/bart/workspace/impera/src/impera/execute/runtime.py", line 50, in set_value
    raise RuntimeException(None, "Value set twice")
impera.ast.RuntimeException: Value set twice (reported in <impera.ast.statements.call.FunctionCall object at 0x7f5281deb160> (./libs/vm/model/_init.cf:42))

The value was set once in the constructor and once in an implementation in a module.

Change name, contact and year in license headers

Migrated from gitlab, originally by @bartv on Apr 25, 2016, 11:34

Update module create command to add module to the project

Migrated from gitlab, originally by @wouterdb on Mar 24, 2016, 16:06

depends on #19

add test cases to parser tests to have 100% code coverage

Migrated from gitlab, originally by @wouterdb on Apr 22, 2016, 10:56

Add authentication to API in server

Migrated from gitlab, originally by @bartv on Apr 18, 2016, 21:38

Add test case to compiler for complex N-M relations

Migrated from gitlab, originally by @wouterdb on Apr 28, 2016, 12:43

Module versioning system should also work for module.yaml

Migrated from gitlab, originally by @wouterdb on Jan 11, 2016, 16:01

Currently this only works for project.yaml

Add list as type for entity attributes

Migrated from gitlab, originally by @bartv on Apr 20, 2016, 08:34

entity Zone:
list slaves
end

Zone(slaves=["192.168.1.1", "192.168.1.2"])

Change all related files to support new version schema

Migrated from gitlab, originally by @bartv on Apr 25, 2016, 13:13

setup.py, debian files, specfile, ...

make non exact versions work correctly once more

Migrated from gitlab, originally by @wouterdb on Apr 15, 2016, 15:21

and add a test case

Handle multiple unkown parameters in forms

Migrated from gitlab, originally by @bartv on Feb 16, 2016, 09:31

[{'source': 'form', 'metadata': {'type': 'form', 'record_id': '84ba03da-949a-4e95-b568-110268487eba'}, 'parameter': 'email'}, {'source': 'form', 'metadata': {'type': 'form', 'record_id': '3d5746c2-22b5-40ea-9b73-1bf439e0826a'}, 'parameter': 'email'}]
ERROR Failed to commit resource updates (An exception occured: ('Tried to save duplicate unique keys (insertDocument :: caused by :: 11000 E11000 duplicate key error index: impera.unknown_parameter.$name_1_environment_1_resource_id_1_version_1 dup key: { : "email", : BinData(3, F558AD021D994496BD28B228E7DFCCBD), : "", : 1455611303 })',))

Serve dashboard from within the API server with a tornado static handler

Migrated from gitlab, originally by @bartv on Apr 18, 2016, 21:38

Clean up proxy.py

Migrated from gitlab, originally by @bartv on Apr 29, 2016, 08:00

Proxy.py line 90 is not up to par with inmanta quality standard :)

Allow to delete an agent

Migrated from gitlab, originally by @bartv on Apr 18, 2016, 21:39

Notify on record change

Migrated from gitlab, originally by @bartv on Dec 4, 2015, 09:54

Enable cross hosts requires again

Migrated from gitlab, originally by @bartv on Dec 3, 2015, 10:39

With the rpc rewrite all communication has to go through the server. Therefore agents are no longer notified of any deploy updates