ai-planning / planutils Goto Github PK
View Code? Open in Web Editor NEWGeneral library for setting up linux-based environments for developing, running, and evaluating planners.
License: MIT License
General library for setting up linux-based environments for developing, running, and evaluating planners.
License: MIT License
There are a variety of ways to use this project, and they should be clearly articulated.
The upgrade script just upgrades the installed planners. When there is a new version of planutils installed (e.g., using pip install --upgrade planutils
), then a planutils setup
is required to create the missing bash scripts.
Fix should be for the upgrade script to create those missing bash script files.
Hello, does any of these planners do numeric planning? I checked downward and related and they do not.
(planutils) root@e502d9d6e5d5:~$ smtplan
Package not installed!
Download & install? [Y/n] y
smtplan will be installed.
About to install the following packages: smtplan (148M)
Proceed? [Y/n] y
Installing smtplan...
FATAL: Unable to get library client configuration: remote has no library client
Error installing smtplan. Rolling back changes...
rm: cannot remove 'smtplan.sif': No such file or directory
(planutils) root@e502d9d6e5d5:~$
(planutils) root@e502d9d6e5d5:~$ singularity remote login DefaultRemote
Generate an access token at https://cloud.apptainer.org/auth/tokens, and paste it here.
Token entered will be hidden for security.
Access Token:
However, going to https://cloud.apptainer.org/auth/tokens times out, as does any other URL from that domain.
Instead of adding to the ~/.bashrc
for getting the path updated, do the following:
planutils run {package}
functionality (closed with #49 )planutils activate
that places adds the generic bin folder to the PATH
variable.docker with the --privileged option
That requirement it's blocking my full use of planutils in heavily containerized contexts.
Not all the tools here require that, but it's friction.
It seems the solution would be to maintain both singularity and docker images. Right? I'd pay the price
We don't want unsecure installation of planners.
See that in apt-get
and other package managers the semantics of user query and forced install are separated:
Currently, install
command lacks this separation. It can choose to force the installation, but its true meaning is yes
.
Example error:
$ planutils run enhsp
FATAL: could not open image /root/.planutils/packages/enhsp-2019/enhsp-2019.sif: failed to retrieve path for /root/.planutils/packages/enhsp-2019/enhsp-2019.sif: lstat /usr/local/lib/python3.6/dist-packages/planutils-0.2.12-py3.6.egg/planutils/packages/enhsp-2019/enhsp-2019.sif: no such file or directory
Docker recipe that can be adapted:
#################################
# Download and Install Pyperplan
#################################
ENV PYPERPLAN_URL=https://github.com/aibasel/pyperplan/archive/master.tar.gz
RUN curl -SL $PYPERPLAN_URL | tar -xz \
&& mv pyperplan-* pyperplan
RUN echo 'alias pyperplan="python3 ${BASE_DIR}/pyperplan/src/pyperplan.py"' >> ~/.bashrc
Hello Christian! Following-up our discussion on the other repository, I tried creating the scripts for an ENHSP package based on your TEMPLATE
scripts. I am not sure if the commands I am executing respect the working directory in which you will be executing the script from (please review that if you can). Also, I am not sure how you will set the variables for setting the domain and problem in the run
script. Here they go:
install.sh (notice that I need sudo
for installing packages and changing permissions)
#!/bin/bash
# To use if root is required
[ "$UID" -eq 0 ] || (echo "installation requires root access"; exec sudo "$0" "$@")
# Install general linux dependencies
sudo apt install openjdk-8-jre openjdk-8-jdk
# General setup / configuration
wget https://gitlab.com/enricos83/ENHSP-Public/-/archive/master/ENHSP-Public-master.zip
unzip ENHSP-Public-master.zip
rm ENHSP-Public-master.zip
mv ENHSP-Public-master/ enhsp
cd enhsp
./compile
./install
sudo chmod +x enhsp
run.sh (you can set the heuristic using the flag -h
)
#!/bin/bash
./enhsp --domain $1 --problem $2
uninstall.sh
#!/bin/bash
rm -rf enhsp/
manifest.json
{
"name": "ENHSP",
"description": "Expressive Numeric Heuristic Search Planner (https://gitlab.com/enricos83/ENHSP-Public)",
"dependencies": []
}
The current check-installed
implementation does only verify if a planner has been installed by the current user:
planutils/planutils/package_installation.py
Lines 31 to 32 in ef05ad6
As the planners are installed globally in /usr/local/lib/python3.7/dist-packages/planutils-0.2.6-py3.7.egg/planutils/packages
, a planner will always been installed for every user, but the corresponding name is only inserted in the settings.json of the user, who issued the install command.
I do suggest either move the settings.json to the global directory (at least the installed
part) or add individual "check-installed" scripts to the planner folders in /usr/local/lib/python3.7/dist-packages/planutils-0.2.6-py3.7.egg/planutils/packages
,
e.g. [...]/planutils/packages/enhsp-2020/check-install
#!/bin/bash
test -f "$(dirname $0)/enhsp-2020.sif"
exit $? #Return return code of test
PS: As installing planners requires root permissions, but executing is file with user permissions, I had to modify my setting.json manually at the moment.
While testing, I was confused why my changes weren't reflected by building the default Dockerfile. An additional Dockerfile with a COPY
instruction rather than a pypi install would be helpful. Arguably, this could be a better default build because you are cutting out the pypi middleman.
Should this go in the "environments" directory?
This is different than #9 because it specifically focuses on planutils. I'm not sure what more is needed in the environment to cover #9
Can use this as the latest binary to extract: https://dev.azure.com/schlumberger/4e6bcb11-cd68-40fe-98a2-e3777bfec0a6/_apis/build/builds/48/artifacts?artifactName=linux64&api-version=5.1&%24format=zip
planutils --remove planner
Since singularity seems to be the leading virtualization technique for high-performance planning, and since the original hub has been discontinued, it's probably worth our while to have a community-led effort to host planners and planning-related software somewhere. Ala https://singularityhub.github.io/sregistry/
@FlorianPommerening : This something you guys would be interested in maintaining from Basel?
Would require some way for the package to store the prefix location so the python scripts can find it.
To make sure we don't break something, there should be some recipe to test each and every package.
Perhaps a test
script in the package folder that prints ok
or anything else depending.
Idea is that we'd make sure every package has their test
file run cleanly before merging.
Environment to setup things for running evaluations (benchmarks, lab, etc).
planutils remote ...
should operate the same as planutils run ...
, except call the planner-as-a-service server (only working for those packages that are deployed).
Trying to build the docker image with:
docker build -t planutils:latest .
I get the following error:
#6 2.523 27000K .......... .......... .......... .......... .......... 99% 14.1M 0s
#6 2.526 27050K ...... 100% 276M=1.7s
#6 2.526
#6 2.526 2022-07-05 15:15:43 (15.8 MB/s) - 'apptainer_1.0.2_amd64.deb' saved [27705364/27705364]
#6 2.526
#6 2.538 dpkg: error processing archive apptainer_1.0.2_amd64.deb (--install):
#6 2.538 package architecture (amd64) does not match system (arm64)
#6 2.542 Errors were encountered while processing:
#6 2.542 apptainer_1.0.2_amd64.deb
------
executor failed running [/bin/sh -c wget https://github.com/apptainer/apptainer/releases/download/v1.0.2/apptainer_1.0.2_amd64.deb && dpkg -i apptainer_1.0.2_amd64.deb && rm apptainer_1.0.2_amd64.deb]: exit code: 1
Arrêt du processus de terminal "/bin/zsh '-c', 'docker build --pull --rm -f "Dockerfile" -t planutils:latest "."'". Code de sortie : 1.
I am working on a Mac M1, hence it works with arm64 instead of amd64.
While running delfi I bumed in this problem:
shell > delfi Examples/blocks-domain-complete.pddl Examples/blocks-problem1.pddl
Traceback (most recent call last):
File "/workspace/delfi/plan-ipc.py", line 13, in <module>
from dl_model import selector
File "/workspace/delfi/dl_model/selector.py", line 4, in <module>
from keras.models import model_from_json
File "/usr/local/lib/python2.7/dist-packages/keras/__init__.py", line 3, in <module>
from . import utils
File "/usr/local/lib/python2.7/dist-packages/keras/utils/__init__.py", line 2, in <module>
from . import np_utils
File "/usr/local/lib/python2.7/dist-packages/keras/utils/np_utils.py", line 6, in <module>
import numpy as np
File "/home/marco/.local/lib/python3.8/site-packages/numpy/__init__.py", line 132
raise ImportError(msg) from e
It seems that it tries to use my local version of numpy instead of using the one in the local internal to the singularity/docker image. TO have it running I had to unset the PYTHONPATH environment variable. May be one can specify to then run in a fresh environment by passing -e flag to the run singularity command i.e. singularity run -e $(dirname $0)/delfi.sif $@
Should be minimal and reflect the vagrant one.
Idea would be to lean on a pre-existing package manager to deal with the web of dependencies, versioning, etc.
(from @jendrikseipp )
Install planutils
as a thin wrapper to Anaconda:
pip install planutils
Install conda, create and activate new environment into which packages will be installed, add conda package repository with conda packages for planners:
planutils setup
Ask user whether to install lama conda package, then possibly run lama:
lama domain.pddl problem.pddl
Added benefits of having minimal impact on .bashrc
edits and a sandboxed environment for the planning stuff.
Both upgrade and multi-package installation might be broken if the order comes in wrong. Instead, it should be re-ordered to follow a topological sort. Also need to consider what to do with cyclic dependencies (if we choose to allow them).
How to reproduce:
environments/server/setup.sh
docker run -p 5555:5555 --privileged -d planutils-server
import requests
import time
from pprint import pprint
req_body = {
"domain":"(define (domain BLOCKS) (:requirements :strips) (:predicates (on ?x ?y) (ontable ?x) (clear ?x) (handempty) (holding ?x) ) (:action pick-up :parameters (?x) :precondition (and (clear ?x) (ontable ?x) (handempty)) :effect (and (not (ontable ?x)) (not (clear ?x)) (not (handempty)) (holding ?x))) (:action put-down :parameters (?x) :precondition (holding ?x) :effect (and (not (holding ?x)) (clear ?x) (handempty) (ontable ?x))) (:action stack :parameters (?x ?y) :precondition (and (holding ?x) (clear ?y)) :effect (and (not (holding ?x)) (not (clear ?y)) (clear ?x) (handempty) (on ?x ?y))) (:action unstack :parameters (?x ?y) :precondition (and (on ?x ?y) (clear ?x) (handempty)) :effect (and (holding ?x) (clear ?y) (not (clear ?x)) (not (handempty)) (not (on ?x ?y)))))",
"problem":"(define (problem BLOCKS-4-0) (:domain BLOCKS) (:objects D B A C ) (:INIT (CLEAR C) (CLEAR A) (CLEAR B) (CLEAR D) (ONTABLE C) (ONTABLE A) (ONTABLE B) (ONTABLE D) (HANDEMPTY)) (:goal (AND (ON D C) (ON C B) (ON B A))) )"
}
solve_request=requests.post(f"http://localhost:5555/package/{planner}/solve", json=req_body).json()
print(solve_request)
Returns
{'stderr': '/root/.planutils/packages/dual-bfws-ffparser/run: line 2: lapkt: command not found\n', 'stdout': ''}
The planutils upgrade should be called whenever a new version is upgraded from pip and the like.
Using Docker on a Mac M1 (arm64), I try to run OPTIC:
(planutils) root@8f6fe7a98cf4:~$ optic
Package not installed!
Download & install? [Y/n] y
optic will be installed.
About to install the following packages: optic (297MB)
Proceed? [Y/n]
Installing optic...
INFO: Downloading network image
296.5MiB / 296.5MiB [=================================================================================================================================================================================================================] 100 % 24.6 MiB/s 0s
Finished installing optic (size: 297M)
Successfully installed optic!
Original command: optic
Re-run command? [Y/n]
FATAL: could not open image /root/.planutils/packages/optic/optic.sif: the image's architecture (amd64) could not run on the host's (arm64)
It fails because the binary package downloaded by planutils is for amd64.
Currently it just deletes/installs the package again. We probably want to have packages provide upgrade
scripts, and keep versions in the manifest.
Something like this...
planutils help <tool>
...should spit out a help message defined in the package manifest.
Not sure what the best planner would be, but it's a requested feature for the online solver so we should include one.
master
-> main
Break all the things...
I am trying to see how we could use planners installed by planutils from within VS Code (besides the planning-as-a-service, which will eventually also solve it). Today, the instructions say make two calls:
planutils activate
lama domain.pddl problem.pddl
The activation feels natural for the hands on phase, just after planutils installation, when the environment needs to be setup by a series of command-line calls. But beyond that to be able to interact with the planners from scripts (or from VS Code in this case), could we have a single line syntax such as this?
planutils some-switch lama domain.pddl problem.pddl
I am trying to use planutils, rather than cloning it and building it first. The package appears to be pushed to https://pypi.org/project/planutils/. How do I use it from there? If I figured it out, I would like to update the README.md
and then I would automate the setup by the VS Code extension.
Here is what I tried (and failed):
user@computer:~/c/github/vscode-pddl-samples$ pip install planutils
Collecting planutils
Downloading planutils-0.7.6-py3-none-any.whl (52 kB)
|████████████████████████████████| 52 kB 633 kB/s
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from planutils) (2.22.0)
Installing collected packages: planutils
Successfully installed planutils-0.7.6
user@computer:~/c/github/vscode-pddl-samples$ planutils activate
planutils: command not found
I am on Ubuntu 20.04 running as WSL on Windows.
It would be nice to integrate and made available the optic temporal planner in the planutils framework.
I've finished setting up singularity for optic.
You can find all here: https://github.com/roveri-marco/optic/tree/singularity-build-and-release
waiting to merge in the master branch of the repository. The singularity sif file is created as well with github workflows
and can be downloaded directly from there.
How can I contribute adding also the plautils part? I've downloaded the repo and created a branch, but I'm not able to push.
Environment for setting up things for the development of planners and planning technology.
It would be great if the README had some brief installation instructions. Is it possible to use pipx install planutils
? In which ways is the bash environment altered?
Environment for running planners and planning utilities as a local or remote server.
In order to include planutils commands in scripts, it would be useful to have an optional argument to just auto-respond as "yes".
I'm using the Docker Image from the Docker Hub as mentioned here, I tried to use the smtplan
planner and I have got this error:
(planutils) root@104582ad0e5f:~$ smtplan
Package not installed!
Download & install? [Y/n] Y
smtplan will be installed.
About to install the following packages: smtplan (148M)
Proceed? [Y/n] Y
Installing smtplan...
FATAL: Unable to get library client configuration: remote has no library client
Error installing smtplan. Rolling back changes...
rm: cannot remove 'smtplan.sif': No such file or directory
This issue didn't appear with optic
planner for example. can you please help me resolve this issue? thanks in advance.
Perhaps tied to the issues closed, like the API repo does.
Should be minimal, reflect the docker one, and should be done with the attached files in mind (issues with a Vagrant image used in Basel for teaching).
Thanks for the excellent docker image,
I tried to run the lama example lama domain.pddl problem.pddl
, but I'm getting this error:
root@8155b66bf2c6:~# planutils activate
Entering planutils environment...
(planutils) root@8155b66bf2c6:~$ lama domain.pddl problem.pddl
Package not installed!
Download & install? [Y/n] Y
lama will be installed.
About to install the following packages: downward (36M), lama (20K)
Proceed? [Y/n] Y
Installing downward...
FATAL: While pulling shub image: error fetching image to cache: failed to get manifest for: shub://aibasel/downward: no response received from singularity hub
Error installing downward. Rolling back changes...
rm: cannot remove 'downward.sif': No such file or directory
The steps that I have followed to install the docker image:
$ docker pull aiplanning/planutils
$ docker run -it --privileged aiplanning/planutils bash
$ planutils activate
$ lama domain.pddl problem.pddl
Can you please tell me how to solve this issue please? thanks in advance.
Singularity recipe can be found here:
https://github.com/ronwalf/HTN-Translation/blob/hpddl-stable/deploy/htntranslate-hpddl.def
If the domain or problem file has a leading /
, then it doesn't seem to pass the file in appropriately.
{this}
in the call string.{arg1}-{arg2}
)Fails because
du -sh .
in the package directory (which only works if installing the package stores everything locally), get a measure of how much space was used pre/post installation.A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.