vcstools/rosinstall

Archived

This repository has been archived due to a lack of maintainance and no longer being used in the ROS project.

rosinstall

Command-line tools for maintaining a workspace of projects from multiple version-control systems, tailored for the ROS (Robot operating system) community.

See http://docs.ros.org/independent/api/rosinstall/html

Installing

Install the latest release on Ubuntu using apt-get:

$ sudo apt-get install python-rosinstall

On other Systems, use the pypi package:

$ pip install rosinstall

For Bash/Zsh completion:

$ pip install rosinstall-shellcompletion

To test in a development environment

rosinstall's Issues

rosinstall should create env.sh

For the PR2 and other multi-machine systems that require remote launches, and for catkin compatibility in general, rosinstall should generate an appropriate env-loader script, along with the existing setup.(sh|bash|zsh) scripts.
For the PR2, it would be useful if the setup.sh included a line to set ROS_ENV_LOADER to the location of the env-loader.

https://kforge.ros.org/vcstools/trac/ticket/52

ROS_WORKSPACE unset by groovy setup.sh must be set in rosbuild workspace

Currently is first set then unset by groovy profile in rosws generated setup.sh

rosco fails for wge100_driver and possibly others

It looks like the API yaml for packages and stacks has moved from stack.yaml to manifest.yaml, and rosco needs to be updated for the new location.

$ rosco wge100_driver
error contacting http://ros.org/doc/api/wge100_driver/stack.yaml:
HTTP Error 404: Not Found
cannot locate information about wge100_driver: HTTP Error 404: Not Found

new rosws detect command

I suggest a new command
rosws detect [PATH]

which craws the workspace / path argument for folders, looking for new SCM subfolders or changed state, to include those in the .rosinsall file.
As such, it is a wrapper to the rosws set command, and could also be implemented as
rosws set --detect
or something similar (open to suggestions).

The command need to detect the presence of a SCM marker fodler/file, then compare this with the .rosinstall file, and if not like this in the .rosinstall, suggest a fix of the rosinstall file. This can be an ambiguous operation, e.g. the new state can be a new tag or new branch, so the user has to decide which state to capture, meaning the operation would by default be interactive (switches can exist to prefer some operation or just fail in case of ambiguity).

the use case is that a user performs changes to his workspace using native scm commands, like git clone, git checkout, etc., and wants the new workspace layout to be reflected in his .rosinstall file.

I can work on the details and an implementation if others agree to this enhancement.

Support HTTP authentication when specifying .rosinstall file

Suggest some combination of the following:

• Interactively prompt for user/pass as necessary.
• Accept --username, prompt for password.
• Accept http://user:pass@server/foo.rosinstall format.

Would be useful for internal development, unreleased software, etc.

rosinstall should retry on login failures to remote repos (ros ticket #3501)

{{{
Installing ... to /wg/stor5_home4/watts/scratch/dback_src/
http authorization required
user: watts
password:
abort: authorization failed
ERROR: Failed to install tree '/wg/stor5_home4/watts/scratch/dback_src'
}}}

This should allow me to retry, with a prompt for (abort/fail/retry/continue anyway) after a bad password or other failure.

trac data:

• Owner: tfoote
• Reporter: watts
• Reported at: Wed May 25 14:02:42 2011
• URL: https://code.ros.org/trac/ros/ticket/3501

rosws doesn't respect ROS_PACKAGE_PATH from a setup file

Follow these directions: http://www.ros.org/wiki/groovy/Installation/Source

And try to use the setup.bash file made by rosws instead of manually setting the ROS_PACKAGE_PATH before the rosmake and the ROS_PACKAGE_PATH will be empty.

invalid PYTHONPATH extension in fuerte

Generated setup.sh includes the line:
export PYTHONPATH=$ROS_ROOT/core/roslib/src:$PYTHONPATH
which only seems to be a valid path in pre-fuerte.
Should we protect this and only execute if path exists?

https://kforge.ros.org/vcstools/trac/ticket/60

rosinstall --generate-versioned-rosinstall error with other tag (ros ticket #3735)

rosinstall --generate-versioned-rosinstall outputs ERROR as followings:
{{{
$ rosinstall --generate-versioned-rosinstall=latest.rosinstall .
rosinstall operating on /home/k-okada/prog/rtm-ros-robotics from specifications in rosinstall files /home/k-okada/prog/rtm-ros-robotics/.rosinstall
ERROR: Cannot generate versioned outputs with non source types
}}}

The ERROR comes from following line in the .rosinstall file
{{{

• other: {local-name: /opt/ros/electric/ros}
• other: {local-name: /opt/ros/electric/stacks}
  }}}

For my usecase, it would be nice if the program just print WARNING and output with comment out these lines.

trac data:

• Owner: tfoote
• Reporter: k-okada
• Reported at: Wed Nov 2 20:43:02 2011
• URL: https://code.ros.org/trac/ros/ticket/3735

new dropin element for .rosinstall files

Currently rosws allows "other" elements which will only be added to the ROS_PACKAGE_PATH by the setup.sh rosws / rosinstall generates. These elements have no function in catkin (using wstool).

Catkin allows drop-in semantics by default, meaning users are free to git clone catkin packages into their source space, and catkin will build them. With rosws+rosbuild, this behavior only works in sandbox folders, in which no VCS operations happen.

In vcstools/vcstools#27 I explored the idea of going less stateful, but the idea there is undesirable in my opinion for the reasons given there.

So another idea is to introduce a new rosnstall syntax element "dropin", which in rosws also adds the folder to the ROS_PACKAGE_PATH, but also performs default scm operations the way Dirk suggests for a generic vcstool command.

This would allow in catkin to define dropin folders where people can clone into, getting both catkin immediate build as well as immediate support for vcs operations via wstool.

The semantics would have to be slightly different for git than for the other SCMs, because
git co tag123
git update
remains at tag123, whereas similar operations for svn, hg and bzr would on the second call move away from that revision, if possible.

A Caveat here is that for catkin, this features does not fully provide what Dirk imagines, being able to create a nice workspace layout via native SCM operations and then export that layout to a .rosinstall file (assuming the additional hierarchy of a dropin subfolder is not "nice"). However we could make sure that if a user insists, we allow "." as a valid value for the dropin element, making the whole workspace a drop-in folder for vcs operations.

There are some other corner cases with difficulties, such as folders having multiple scm folders, where I would suggest just creating a warning.

Also given the choice of making the whole workspace folder a dropin folder, some users may want a feature to exclude subfolders from vcs operations, just like the CATKIN_IGNORE file. So to mirror that we could have a WSTOOL_IGNORE file support as well, for dropin folders.

Changing the behavior of the "other" element to also perform vcs operations with default semantics on any scm subfolder is probably not a good idea.

roslocate does not handle info correctly anymore

This is due to updated indexer metadata storage.

From ros/rosdistro/issues/366

rosinstall --generate-versioned-rosinstall broken

Ever since the refactoring rosinstall 0.6, the --generate-versioned-export option was broken. It only exported what was in the .rosinstall file to start with.

rosws overwrites ROS_PACKAGE_PATH set by setup-file

up to groovy, this was fine, since we only cared about ROS_ROOT, PATH and PYTHONPATH. However if we want to rely on the RPP set by catkin, we have to find a fix.
Simply prepending may not work, since in the pre-fuerte case this would generate some duplicate entries. Seem harmless in fuerte, need to check electric...

raw output for diff and stat

As requested by @dirk-thomas , rosws commands should allow getting "raw" output from the diff and status commands. Currently rosws (with the help of vcstools flexibility) defaults to diff and status being relative to the workspace root.

Different SCMs provide different output.

E.g. if the SCM folder is ~/foo, and we are in ~/bar, then a

command scm ../foo

can produce a diff where each path starts with ../foo, or where each path is relative to ~/foo, or plainly reject to create such a 'silly' diff (git).

We can output these reactions as "raw" output, or we could offer a different normalisation where each diff will be relative to the repository root, rather than the workspace root.

The vcstools library currently has a basepath parameter to both the diff and status commands, which defaults to the vcsclient root in the API (should be local repository root). Currently rosws overrides this default giving the workspace root. So the technical change should boil down to adding a --raw option or similar and adapting the call to vcstools.

Pending clarification of "raw" by @dirk-thomas.

rosws init fails when ~/catkin_ws is a symbolic link

This issue is similar to that of issue 22
ROS_PACKAGE_PATH is wrongly created if the workspace directory is a symbolic link

ROS_PACKAGE_PATH=~/catkin_ws/src:/path_to_symbolic_link/src:/opt/ros/groovy/share:/opt/ros/groovy/stacks

This causes the following error

multiple files named [launchfile.launch] in package [package]:
- ~/catkin_ws/src/package/launch/launchfile.launch
- path_to_symbolic_link/src/package/launch/launchfile.launch
Please specify full path instead

roslocate could recursively find sources of dependencies

Based on the data that feeds the ros wiki listing names of dependencies fo packages, roslocate could be extended top optionally list also the sources of dependencies, and with another option excluding any dependencies that are already in found in the current environment. The process can take a while, since it would have to download from the web multiple times.

info table for rosinstall (ros ticket #3688)

Another change for REP 110, the --info command gives an overview of all managed entries.

Columns:
1 Localname: A normalized path (relative or absolute, whatever is shorter)
2. E whether the directory exists
3. SCM type
4. M modified state (whether diff returns anything)
5. Version-spec what .rosistall wants (tag, branch, ...)
6. Version-rev what numeric id the Version spec has (may name Spec-rev?
7. Current-Rev what numeric id the current revision has
8. URI remote repository

Patch attached with unit tests

Not sure about the headers or the entry line, the SCM-like syntax later will have more room for options like --short or --verbose. Unit tests only care about the data.

trac data:

• Owner: kruset
• Reporter: kruset
• Reported at: Sat Sep 24 06:19:35 2011
• URL: https://code.ros.org/trac/ros/ticket/3688

test failure on precise due to no zsh

The tests seem to be assuming zsh is present. But this does not appear to be true on my new precise desktop.

======================================================================
FAIL: test_prereq (test.local.test_rosinstall.RosinstallLocalDistro)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tfoote/work/rosinstall/test/local/test_rosinstall.py", line 169, in test_prereq
    self.assertTrue(os.path.exists('/bin/zsh'))
AssertionError: False is not true

Little typo in `rosws merge --help`

The example command for piping the output of roslocate into rosws merge in the help string is:

$ roslocate info robot_mode | rosws merge -

I think it should be:

$ roslocate info robot_model | rosws merge -

integrate the rosinstall file generator scripts into rosinstall package

The rosdistro module from catkin-debs.buildfarm is needed for wet. And there's a generator for dry as well. I don't believe they add any dependencies, with the possible exception of rospkg, maybe it could just go into rospkg too. We'll have to do a full audit.
This will replace the need for hosting the rosinstall generator script.

debian packages lack bash completion

Currently the ri_bash_completion project on github provides shell scripts to create bash completion. Those should be included in the debians (might not even require separate packages). The pypi version of the completion might also be dropped.

For wstool adapting the script should be straightforward (e.g. removing regenrate command)

If you can tell me how to generate a debian package locally, I am happy to work on that.

rosws init fails when /opt/ros is a symbolic link

In our case, /opt/ros is a symbolic link pointing to /work/ros on a different partition. Calling rosws init fails with the following error:

lohr@atbeetz15:~/rosws$ /usr/bin/rosws init . /opt/ros/fuerte/
Using ROS_ROOT: /work/ros/fuerte/
Writing /usr/wiss/lohr/rosws/.rosinstall
(Over-)Writing setup.sh, setup.bash, and setup.zsh in /usr/wiss/lohr/rosws
ERROR in rosinstall: Multiple ros stacks found in config set(['/opt/ros/fuerte/share/ros', '/work/ros/fuerte/share/ros']), Please elimate all but one.
They come from the following sources: {'/opt/ros/fuerte/setup.sh': '/opt/ros/fuerte/share/ros', '/work/ros/fuerte/share/ros': '/work/ros/fuerte/share/ros'}

My guess is that the expansion of links is inconsistent, i.e. sometimes the absoulte path and sometimes the path including the link is used.

tests failing on lucid

5 tests use subprocess.check_output which doesn't exist in python 2.6. I don't know how the travis builds are passing on 2.6. Also there looks to be a syntax error in scm_test_base.py

I was trying to release but get these errors:

..
======================================================================
ERROR: checkout into temp dir and test setup files
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tfoote/rosinstall/test/network/test_boxturte_install.py", line 70, in test_source
    self.simple_rosinstall)
  File "/home/tfoote/rosinstall/test/scm_test_base.py", line 76, in _create_yaml_file
    content += "- %s:\n" % elt["type"]
TypeError: string indices must be integers, not unicode

======================================================================
ERROR: checkout into temp dir and test setup files
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tfoote/rosinstall/test/network/test_cturtle_install.py", line 77, in test_source
    subprocess.check_output(". %s" % os.path.join(self.directory, 'setup.bash'), shell=True, env=self.new_environ, executable='bash')
AttributeError: 'module' object has no attribute 'check_output'
-------------------- >> begin captured stdout << ---------------------
rosinstall operating on /tmp/tmpI7tXXj from specifications in rosinstall files  http://ros.org/rosinstalls/cturtle_ros.rosinstall
(Over-)Writing /tmp/tmpI7tXXj/.rosinstall
[ros] still active
[ros] still active
[ros] still active
[ros] still active
(Over-)Writing setup.sh, setup.bash, and setup.zsh in /tmp/tmpI7tXXj

rosinstall update complete.


Now, type 'source /tmp/tmpI7tXXj/setup.bash' to set up your environment.
Add that to the bottom of your ~/.bashrc to set it up every time.

If you are not using bash please see http://www.ros.org/wiki/rosinstall/NonBashShells 

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: checkout into temp dir and test setup files
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tfoote/rosinstall/test/network/test_diamondback_install.py", line 75, in test_source
    subprocess.check_output(". %s"%os.path.join(self.directory, 'setup.sh') , shell=True, env=self.new_environ)
AttributeError: 'module' object has no attribute 'check_output'
-------------------- >> begin captured stdout << ---------------------
rosinstall operating on /tmp/tmpFrBMe0 from specifications in rosinstall files  http://packages.ros.org/cgi-bin/gen_rosinstall.py?rosdistro=diamondback&variant=ros-base&overlay=no
(Over-)Writing /tmp/tmpFrBMe0/.rosinstall
[ros,ros_comm] still active
(Over-)Writing setup.sh, setup.bash, and setup.zsh in /tmp/tmpFrBMe0

rosinstall update complete.


Now, type 'source /tmp/tmpFrBMe0/setup.bash' to set up your environment.
Add that to the bottom of your ~/.bashrc to set it up every time.

If you are not using bash please see http://www.ros.org/wiki/rosinstall/NonBashShells 

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: checkout into temp dir and test setup files
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tfoote/rosinstall/test/network/test_electric_install.py", line 75, in test_source
    subprocess.check_output(". %s"%os.path.join(self.directory, 'setup.sh') , shell=True, env=self.new_environ)
AttributeError: 'module' object has no attribute 'check_output'
-------------------- >> begin captured stdout << ---------------------
rosinstall operating on /tmp/tmp7HYhyD from specifications in rosinstall files  http://packages.ros.org/cgi-bin/gen_rosinstall.py?rosdistro=electric&variant=ros-base&overlay=no
(Over-)Writing /tmp/tmp7HYhyD/.rosinstall
[ros,ros_comm,common_rosdeps] still active
(Over-)Writing setup.sh, setup.bash, and setup.zsh in /tmp/tmp7HYhyD

rosinstall update complete.


Now, type 'source /tmp/tmp7HYhyD/setup.bash' to set up your environment.
Add that to the bottom of your ~/.bashrc to set it up every time.

If you are not using bash please see http://www.ros.org/wiki/rosinstall/NonBashShells 

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: checkout into temp dir and test setup files
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tfoote/rosinstall/test/network/test_fuerte_install.py", line 95, in test_source
    subprocess.check_output(". %s" % os.path.join(self.directory, 'setup.sh') , shell=True, env=self.new_environ)
AttributeError: 'module' object has no attribute 'check_output'
-------------------- >> begin captured stdout << ---------------------
rosinstall operating on /tmp/tmpVMeWy2 from specifications in rosinstall files  /tmp/tmpVMeWy2/simple.rosinstall
(Over-)Writing /tmp/tmpVMeWy2/.rosinstall
[dynamic_reconfigure,nodelet_core,xacro,bullet,bond_core,pluginlib,geometry,bfl] still active
[orocos_kinematics_dynamics,common,filters,diagnostics,common_rosdeps,driver_common,executive_smach,robot_model] still active
(Over-)Writing setup.sh, setup.bash, and setup.zsh in /tmp/tmpVMeWy2

rosinstall update complete.


Now, type 'source /tmp/tmpVMeWy2/setup.bash' to set up your environment.
Add that to the bottom of your ~/.bashrc to set it up every time.

If you are not using bash please see http://www.ros.org/wiki/rosinstall/NonBashShells 

--------------------- >> end captured stdout << ----------------------

rosws missing sys import

https://gist.github.com/4130156

add timeout support to rosinstall calls

https://kforge.ros.org/vcstools/trac/ticket/69

We often have our indexer left hanging on a bad checkout. Sometimes a dropped connection, or server hiccup for which rosinstall just keeps waiting, and the job has to be canceled and restarted.
If we could set a reasonable timeout we'd avoid this problem. This will probably be relatively intrusive in the code flow.

A related problem was dicussed here:
https://code.ros.org/trac/ros/ticket/1780
I think:
We could have something similar to svn --non-interactive
When non-interactive, all SCMs make their commands using --non-interactive options if available
When we run as non-interactive, we also define a timeout on stdout activity.
This also means it should be possible to define a longer timeout than the defaultusing a second option
comment:2 Changed 5 days ago by kruset

However, I just created a change that causes mercurial updates to run without capturing stdout. This might prevent waiting for timeouts on stdout activity.
comment:3 Changed 5 days ago by tfoote

The non-interactive would be good too. But I was more thinking just a pure timeout. The issue we're having in the indexer is that the checkout basically fails part way through.
It looks like calling communicate() in a separate thread is about the best solution: http://stackoverflow.com/questions/1191374/subprocess-with-timeout
Last edited 5 days ago by tfoote (previous) (diff)
comment:4 Changed 4 days ago by kruset

Ok, so this is quite tricky, because we have several requirements.
We want to display the output of commands.
We often want to capture the output of commands as well.
We want to terminate a process properly.
I struggled with calling communicate in a separate thread respecting all of the above, and the code got really messy.
So instead I went for the signal approach, and it seems towork. But I would prefer this to be well code-reviewed. Also I cannot tell whether this runs on windows (or whether it at least runs in the non-timeout case). Attaching patch.
Changed 4 days ago by kruset

attachment timeout.patch added

rosinstall tests are fragile to ordering changes

rosinstall/test/local/test_diff_functions_multi.py is failing 12 tests at the moment which mostly appear to be ordering issues.

For example:

======================================================================                                                                      
FAIL: Test status output for svn when run outside workspace                                                                                 
----------------------------------------------------------------------                                                                      Traceback (most recent call last):                                                                                                            File "/tmp/rosinstall/test/local/test_diff_functions_svn.py", line 203, in test_Rosinstall_status_svn_untracked                               self.assertEqual('?       clone/added-fs.txt\nA       clone/added.txt\nD       clone/deleted.txt\n!       clone/deleted-fs.txt\nM       clone/modified.txt\n', output)                                                                                                              AssertionError: u'?       clone/added-fs.txt\nA       clone/added.txt\nD       clone/deleted.txt [truncated]... != u'?       clone/added-fs.txt\nD       clone/deleted.txt\n!       clone/deleted-f [truncated]...                                                                        ?       clone/added-fs.txt                                                                                                                
- A       clone/added.txt                                                                                                                   
  D       clone/deleted.txt                                                                                                                 
  !       clone/deleted-fs.txt                                                                                                              
  M       clone/modified.txt                                                                                                                
+ A       clone/added.txt  

detect ROS_ROOT in groovy setup files.

ros/catkin#229

instructions for adding your workspace to your .bashrc are incorrect when not in home directory

The quoted 'source groovy/setup.bash' should be made absolute.

tfoote@stitch:/tmp$ rosws init groovy /opt/ros/groovy/
(rosws and py-rosws are experimental scripts, please provide feedback to [email protected])
Using ROS_ROOT: /opt/ros/groovy/
Writing /tmp/groovy/.rosinstall
(Over-)Writing setup.sh, setup.bash, and setup.zsh in /tmp/groovy

rosws init complete.

Add 'source groovy/setup.bash' to the bottom of your ~/.bashrc to set it up every time.

If you are not using bash please see http://www.ros.org/wiki/rosinstall/NonBashShells 
tfoote@stitch:/tmp$ 

rosws does not append to ROS_PACKAGE_PATH but replaces it

I thought rosws was supposed to handle wet overlays

e.g:

source /opt/ros/groovy/setup.sh
And ROS_PACKAGE_PATH=/opt/ros/groovy/share:/opt/ros/groovy/stacks

In the same terminal: mkdir /tmp/ws && cd /tmp/ws && rosws init && source setup.bash
And ROS_PACKAGE_PATH=

I tried it with the latest on two different machines and two different users.

rosinstall: need a way to force install, even with bad server cert (ros ticket #3169)

mit-ros-pkg has a self-signed certificate, which means that SVN creates a prompt for checkout. You can override this in svn with '--non-interactive --trust-server-cert'. This doesn't seem like the right default option, but it would be nice to have an option in rosinstall that forces this. Otherwise we cannot rosdoc mit-ros-pkg w/o manually checking it out on every build machine

trac data:

• Owner: tfoote
• Reporter: kwc
• Reported at: Wed Dec 1 19:21:31 2010
• URL: https://code.ros.org/trac/ros/ticket/3169

rosws: tutorial still recommends using pip to install

Ubuntu users should apt-get python-rosinstall, instead.

Using pip is dangerous because users end up with stale versions.

rosws commit / push commands

It would be convenient to add commit and push commands to rosws.
commit: goes through each repo listing file statuses that would be committed and prompts the user for a commit message
push: pushes each repo

https://kforge.ros.org/vcstools/trac/ticket/62

rosco can't check out groovy pr2_calibration

I suspect there is at least one deeper problem here; either rosco failing to parse the distribution files properly, or missing distribution data for pr2_calibration.

$ rosco pr2_calibration --distro groovy
Traceback (most recent call last):
  File "/usr/bin/rosco", line 136, in <module>
    rosco_main()
  File "/usr/bin/rosco", line 125, in rosco_main
    yaml_string = _read_yaml_from_ros_server(name, options.distro, options_to_branch(options))
  File "/usr/bin/rosco", line 84, in _read_yaml_from_ros_server
    return get_rosinstall(name, data, type_, branch)
  File "/usr/lib/pymodules/python2.7/rosinstall/locate.py", line 59, in get_rosinstall
    raise InvalidData("rosinstall control information for %s %s\n" % (type_, name))
rosinstall.locate.InvalidData: rosinstall control information for package pr2_calibration

add --untracked option for 'rosws info' and / or make it the default behavior

'rosws info' would not indicate any changes for repositories that have untracked files. This is misleading and one have to use 'rosws status --untracked'.

In my opinion, rosws with no parameters should indicate any dirty state, including untracked files. It would be nice to have it as a default.

'python-rosinstall' has no installation candidate for Ubuntu Raring

I installed a fresh copy of Ubuntu Raring on my laptop.

I can't find python-rosinstall in either the ros or ros-shadow-fixed repositories for Raring on packages.ros.org.

prompt_merge refactor

Note to myself:
code in multiproject_cli.prompt_merge() should be improved by avoiding to construct a new config element on each interaction with the user. This was done mostly to avoid deep-copy of the original config element.

One consequence is that currently we read the same yaml from a web location in every iteration. As users probably don't use the extended features of merges very often this has low priority.

roslocate broken

I've identified two issues which broke roslocate functionality.

Firstly, a recent commit ignores the one expected error when querying a stackage:
bc58845#commitcomment-2765099

Secondly, autogeneration of rosinstall file (for packages that don't provide one) produces malformed files:
ce90183#commitcomment-2765074

As far as I can tell, the second issue is addressed in #67. And on the second look, the first issue is also handled, by querying only one url (manifest.yaml).
The following comment was left in locate.py

#1 error is expected when we query package

which is now misleading.

These are also the reasons of rosco failure #72.

old fuerte rosinstall file fails

The .rosinstall file located at http://ros.org/rosinstalls/fuerte-ros-base.rosinstall (as well as the full install) fail due to listing an out-of-date repository listing and out-of-date versions.

introduce include elements

https://kforge.ros.org/vcstools/trac/ticket/8

from http://code.ros.org/lurker/message/20111023.065813.7841f236.gl.html
The change introduces "- include" elements for rosinstall files requiring a URI. A rosinstall file at that location will re read and used as if the elements were in place of the include. The resulting rosinstall file will still only have the "include" element, so no merge with the referenced .rosinstall file happens.
No command-line option exists for this use case, this can only be achieved by manually changing the .rosinstall or merging with a rosinstall having .include elements.
--- a/scripts/rosinstall
+++ b/scripts/rosinstall
@@ -230,17 +230,19 @@ class Config:

     # Get the version and source_uri elements
     source_uri = v.get('uri', None)
     version = v.get('version', '')

     #compute the local_path for the config element
     local_path = os.path.normpath(os.path.join(self.base_path, local_name))

•    if k == 'other':
  

•    if k == 'include':
  

•      self.load_yaml(rosinstall.helpers.get_yaml_from_uri(source_uri),rosinstall_source_uri)
  

•    elif k == 'other':
  

Changed 10 months ago by kruset

Comment:
Maybe the local-name could be used as a subfolder rather than being ignored when given.
Two other robustness issues are likely:

1. Prevent infinite recursion, as you can use a local filename as URI (though infinite recursions are already possible with rosinstall as it is, e.g. "$ rosinstall . . .")
2. Rewriting, when you merge a rosinstall file into your current environment. I think it might only be necessary to extend the list in def rewrite_included_source(source_yaml, source_path):
   There also is a small usability issue on having multiple ros distros configured in the current environment, rosinstall neither checks nor prevents that. And this is more likely to occur than now when an included remote rosinstall file was changed.
   The difference between merging with a rosinstall file and including it could become confusing for new rosinstall users. But personally I never liked merging of rosinstall files much. (Especially the peculiarity that when pointing to a local directory in "$ rosinstall . /path/to/local/dir", and there was a .rosinstall there, the entries would be merged as "other" elements.)
   Last edited 10 months ago by kruset (previous) (diff)
   comment:2 Changed 10 months ago by kruset

def rewrite_included_source(source_yaml, source_path):
#print "before", source_yaml
for entry in source_yaml:

• types = ['svn', 'bzr', 'hg', 'git', 'other']
• types = ['svn', 'bzr', 'hg', 'git', 'other', 'include']
  For better or worse, the change above causes "include" snippets in ".rosinstall" files in directories to merge with to be rewritten as "other" elements. Not sure what the consistent implementation would be.
  Maybe parsing the referenced rosinstall and including all its elements as "other" elements, recursive for "include" elements.
  Last edited 10 months ago by kruset (previous) (diff)
  comment:3 Changed 10 months ago by kruset

diff --git a/scripts/rosinstall b/scripts/rosinstall
--- a/scripts/rosinstall
+++ b/scripts/rosinstall
@@ -213,34 +213,38 @@ class Config:
self.load_yaml(self.source, self.source_uri)
self.valid = True
else:
self.valid = False

def is_valid(self):
return self.valid

• def load_yaml(self, y, rosinstall_source_uri):
• def load_yaml(self, y, rosinstall_source_uri, prefix_path = ''):
  for t in y:
  for k, v in t.iteritems():

 # Check that local_name exists and record it
 if not 'local-name' in v:
   raise ROSInstallException("local-name is required on all rosinstall elements")
 else:
   local_name = v['local-name']

 # Get the version and source_uri elements
 source_uri = v.get('uri', None)
 version = v.get('version', '')

 #compute the local_path for the config element

•    local_path = os.path.normpath(os.path.join(self.base_path, local_name))
  

•    local_path = os.path.normpath(os.path.join(self.base_path, prefix_path, local_name))
  

•    if k == 'other':
  

•    if k == 'include':
  

•      y = rosinstall.helpers.get_yaml_from_uri(source_uri)
  

•      if y != None:
  

•        self.load_yaml(y, rosinstall_source_uri, local_name)
  

•      else:
  

•        raise ROSInstallException("Failed to include uri %s" % source_uri)
  

•    elif k == 'other':
  

  this way local-name is being used as subfolder. local-name: "" also works. Also more robust against bad uris.
  Last edited 10 months ago by kruset (previous) (diff)
  comment:4 Changed 10 months ago by kruset

Also note --diff, ---status and --info (as per uncommited patch) seem to work fine with include elements so far.
comment:5 Changed 10 months ago by DanielStonier

A possible corner-case for subfolder usage - does it work with the electric bare-bones generated .rosinstall (specifically thinking of an include with the 'special' ros stack inside)?
include:
uri: http://packages.ros.org/cgi-bin/gen_rosinstall.py?rosdistro=electric&variant=ros-base&overlay=no
local-name: bare-bones
comment:6 Changed 5 months ago by bit_pirate

Bump!
What's the status of this? Are there still open issues?
It would be really nice to have this feature.
comment:7 Changed 5 months ago by tfoote

In our recent restructuring there were several issues identified which could be a problem for this element.
Consistency of storage is a bit of an issue, especially for use cases like the included file removes an element. This will likely need a non trivial extension to the file format.

python-rosinstall requires python-vcstools but it's not an apt dependency

If you sudo apt-get install python-rosinstall and don't already have python-vcstools installed on your machine, running rosinstall gives you this error:

ERROR Cannot find rosinstall libraries, check your installation. One frequent cause is that rosinstall 0.5 is still installed in /usr/local/lib. Check the rosinstall wiki for solutions.
cannot import name get_vcs_client

which is mighty confusing. Surely python-vcstools should be an official apt depedency?

rosco attempts multiple checkouts

When running rosco, it runs the checkout command twice; for most version-control systems, the second checkout fails and causes rosco to output an error message.

Example:
$ rosco pr2_calibration
('hg', 'pr2_calibration', 'https://kforge.ros.org/calibration/pr2_calibration', 'default')
/usr/lib/pymodules/python2.7/vcstools/vcs_abstraction.py:88: UserWarning: Class VcsClient is deprecated, use get_vcs_client() instead
warnings.warn("Class VcsClient is deprecated, use get_vcs_client() instead")
requesting all changes
adding changesets
adding manifests
adding file changes
added 352 changesets with 1776 changes to 956 files (+6 heads)
updating to branch default
250 files updated, 0 files merged, 0 files removed, 0 files unresolved
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
('hg', 'pr2_calibration', 'https://kforge.ros.org/calibration/pr2_calibration', 'default')
Error: cannot checkout into existing directory

Add support for wildcards to rosws update

https://kforge.ros.org/vcstools/trac/ticket/67

It would be cool to have wildcards in rosws up, e.g.:
rosws update 'cram*'
Tully, what do you think?

Created attached patch, won't push until Tully gives okay for this. (note trac email notification still do not work for me, or Lorenz)
Changed 7 days ago by kruset

attachment wildcards.patch added
comment:2 Changed 7 days ago by tfoote

I'm a little worried about this possibly being a problem in the future. If you don't quote it the base completion will do similar things.
I tested and svn doesn't do this. Does git or hg support wildcard syntax?
comment:3 Changed 6 days ago by moesenle

I don't think version control systems provide wildcard syntax. I guess they rely on the shell's glob expansion which course works for rosws, too. But rosws can be invoked basically everywhere and infers the workspace from the ROS_WORKSPACE variable. Wildcard support is just pure convenience but I felt like it would help me in my everyday workflow :)

setup.sh hickup when chaining workspaces

To reproduce, using 0.6.23:

rosws init foows /opt/ros/fuerte
rosws init barws
cd foows/
mkdir baz
rosws set baz
cd ../barws/
echo "- setup-file: {local-name: ../foows/setup.sh}" >> .rosinstall
. setup.bash 

missing options in docs

as reporded by wjwwood, rosinstall docs miss -- options for rosinstall and rosws.

unit tests rely on >=2.7 logic

These aren't available on lucid.

======================================================================
ERROR: test_version (test.local.test_cli.GetVersionTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tfoote/rosinstall/test/local/test_cli.py", line 56, in test_version
    self.assertIsNotNone(rosinstall.multiproject_cmd.cmd_version())
AttributeError: 'GetVersionTest' object has no attribute 'assertIsNotNone'

======================================================================
ERROR: Failure: AttributeError ('module' object has no attribute 'skipUnless')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/loader.py", line 390, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 86, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/tfoote/rosinstall/test/local/test_rosinstall.py", line 168, in <module>
    class RosinstallLocalDistro(AbstractRosinstallBaseDirTest):
  File "/home/tfoote/rosinstall/test/local/test_rosinstall.py", line 174, in RosinstallLocalDistro
    @unittest.skipUnless(os.path.isdir('/opt/ros/cturtle'),
AttributeError: 'module' object has no attribute 'skipUnless'

======================================================================
ERROR: Failure: AttributeError ('module' object has no attribute 'skipIf')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/loader.py", line 390, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/pymodules/python2.6/nose/importer.py", line 86, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/tfoote/rosinstall/test/local/test_setupfiles.py", line 127, in <module>
    class Genfiletest(AbstractRosinstallBaseDirTest):
  File "/home/tfoote/rosinstall/test/local/test_setupfiles.py", line 150, in Genfiletest
    @unittest.skipIf(HAS_PYTHON3, "No Python3 on this machine")
AttributeError: 'module' object has no attribute 'skipIf'

old git versions allow self cloning from non-existing directory

Reported with git version 1.7.4.1 (Ubuntu natty)

$ cd /tmp
$ git clone foo foo

Cloning into foo...
done.
warning: You appear to have cloned an empty repository.

This behavior can in corner cases prevent the user from getting an appropriate error message from rosws/rosinstall when he mistyped the uri for a git repo.

Suggested fix1: if first arg to git clone would be a path (abs or rel), instead use file://path
However this approach modifies the url stored for remote origin
Suggested fix2: if first arg is a path check manually whether it is also target and raise valueError.
Alternative 3: Don't do anything, probably nobody will ever notice.

Not sure which git version fixes this anyway. 1.7.4.1 broken, 1.7.9.5, fixed, did not see anything in release notes.
EDIT git 1.7.5.4 (oneiric) fixed.

Thoughts?

`rosws merge -` broken

Merging doesn't seem to support input from stdin anymore.

roslocate info <some package> | rosws merge -

This results in a backtrace:

Traceback (most recent call last):
  File "/usr/bin/rosws", line 64, in <module>
    sys.exit(rosinstall.rosws_cli.rosws_main(sys.argv))
  File "/usr/lib/pymodules/python2.7/rosinstall/rosws_cli.py", line 440, in rosws_main
    return ws_commands[command](workspace, args)
  File "/usr/lib/pymodules/python2.7/rosinstall/multiproject_cli.py", line 277, in cmd_merge
    pipedata = "".join(sys.stdin.readlines())
NameError: global name 'sys' is not defined
close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr

I tested with the current version from debian packages and from pip.

Missing elements in ROS_PACKAGE_PATH when chaining workspaces by adding setup-file elements of other workspaces

I am trying to chain rosws workspaces by manually including setup-file elements in the config of workspace foo. According to tkruse`s comment on issue #6 this should be a valid solution to the missing include element issue.

Here is a simple test case, which was copied from issue #55 with an additional other-element in workspace barws:

rosws init foows /opt/ros/fuerte
rosws init barws
cd foows/
mkdir baz
rosws set baz
cd ../barws/
echo "- setup-file: {local-name: ../foows/setup.sh}" >> .rosinstall
rosws set bar_package -y             # <-- This line was added
. setup.bash

Result:

echo $ROS_PACKAGE_PATH
:/tmp/foows/baz:/opt/ros/fuerte/stacks:/opt/ros/fuerte/share:/opt/ros/fuerte/share/ros

bar_package is not contained in the package path and compared to the result of sourcing foows/setup.bash there is an additional leading colon. I guess the setup file from foows overwrites the ROS_PACKAGE_PATH from barws, no matter at which position the config elements stand in the .rosinstall file.

I am using version 0.6.24 of rosws installed as debian package.

create symlink to current toplevel.cmake

Instead if generating a self-made CMakeLists.txt for the root of a catkin workspace rosinstall should use the catkin script "catkin_init_workspace" to symlink (or is that is not possible copy) the toplevel.cmake provided by catkin.
If catkin is not in the environment it should be looked up in the workspace in the subfolder catkin.
If it was not found in both location no CMakeLists.txt should be generated.

Copied from
https://kforge.ros.org/vcstools/trac/ticket/71

Document "tar" option

If "tar" is a valid rosinstall option that will be supported going forward, it should probably be documented here: http://www.ros.org/doc/api/rosinstall/html/rosinstall_file_format.html

Currently, it is not.