bloomberg/phabricator-tools

Phabricator Tools

phabricator-tools's Issues

Arcyd-tester: push_new_review_branch does not handles slashes in filenames

push_new_review_branch() writes a file corresponding to each branch and the name of the file is obtained simply by appending the name of branch to the filepath. This logic fails when the name of branch itself has slashes which is a valid case because the function treats it as a directory and hence fails while trying to create the file.

This is a relatively minor issue since this code is just used by the tester and not actually by Arcyd but thought it will still be nice to at least have it documented here.

static_tests.sh fails to run correctly

Seems that sfood did not like the multiline string it was passed, when I run it I get thrown on to pager. Haven't found where yet. After I quit it I am back at the prompt.

$ lsb_release -a | grep Release
    Release:    12.04
$ uname -a
    Linux os-compaq 3.2.0-48-generic #74-Ubuntu SMP Thu Jun 6 19:45:16 UTC 2013 i686 i686 i386 GNU/Linux
$ bash --version
    GNU bash, version 4.2.25(1)-release (i686-pc-linux-gnu)

I have a patch that I have tested and verified on my system. Have pushed it to branch fix_static_tests branch of my fork

arcyon task-create/update fails to identify project

Running the following command:

arcyon task-create --uri URI --user USERNAME --cert CERT --projects '#PROJECT' --format-id TASK-ID
arcyon task-update --uri URI --user USERNAME --cert CERT --projects '#PROJECT' --format-id TASK-ID

Result in the following error:

Traceback (most recent call last):
File "./arcyon", line 31, in
sys.exit(aoncmd_arcyon.main())
File "/disk1/git/phabricator-tools/py/aon/aoncmd_arcyon.py", line 97, in main
return args.func(args)
File "/disk1/git/phabricator-tools/py/aon/aoncmd_taskupdate.py", line 160, in process
project_to_phid = phlcon_project.make_project_to_phid_dict(conduit)
File "/disk1/git/phabricator-tools/py/phl/phlcon_project.py", line 72, in make_project_to_phid_dict
return {i.name: i.phid for i in query_all(conduit)}
File "/disk1/git/phabricator-tools/py/phl/phlcon_project.py", line 57, in query_all
items = query_some(conduit, window_size, 0)
File "/disk1/git/phabricator-tools/py/phl/phlcon_project.py", line 45, in query_some
results = [QueryResponse(**r) for phid, r in response.iteritems()]
File "/disk1/git/phabricator-tools/py/phl/phlsys_namedtuple.py", line 111, in make_instance
raise Error("missing attributes: " + str(missing_attr))
phlsys_namedtuple.Error: missing attributes: set(['phid', 'name'])

I haven't dug into the code yet, but my brief glance at the docs and conduit suggested that the calls above ought to work. Am I missing anything obvious here?

Arcyd: Fix smoke tests

smoke-tests fail are flaky and often fail on local machines. Following error message is thrown by phabricator:

You can not access the user inside the implementation of a  method which does not require authentication (as per shouldRequireAuthentication()).'

The error is probably by virtue of arcyd running in a multiprocess environment because arcyon works fine when ran as a single process but fails when several instances are ran together in parallel.

Consider upgrading to new Travis infrastructure

The new infrastructure is container bases and is more powerful and fast. Guide for upgrading: http://docs.travis-ci.com/user/migrating-from-legacy/

Arcyd: 'restart' can be interrupted, leaving arcyd offline

In the case where a user becomes impatient and uses Ctrl-C to abort, Arcyd will most likely stop and not be restarted.

Potentially we could not use the var/command/killfile scheme in this case and perhaps use a var/command/reloadfile scheme, so that the running daemon can have the responsibility of both stopping and starting.

arcyon task-query seems to have a bug.

Even more stuff:

arcyon task-query --cert $CERT --uri $URI --user $USER

Fails with the following traceback (most recent call last):
File "./arcyon", line 31, in
sys.exit(aoncmd_arcyon.main())
File "/disk1/git/phabricator-tools/py/aon/aoncmd_arcyon.py", line 97, in main
return args.func(args)
File "/disk1/git/phabricator-tools/py/aon/aoncmd_taskquery.py", line 247, in process
r['statusName'] = phlcon_maniphest.STATUSES[int(r['status'])]
ValueError: invalid literal for int() with base 10: 'open'

This is running against the latest Phabricator.

REQUEST: Add support for project hashtags in 'arcyon task-create --projects ...'

Just a humble suggestion :)

It would be very, very nice if you could use a project's hashtag(s) when creating a task via arcyon task-create.

I've also noticed that the workflow requires that you specify a project name with precise casing. I have a project called Gator, and the command fails if I pass --projects gator.

Thanks for making this FANTASTIC, FANTASTIC tool!

Arcyd: var/log/debug rotates pretty fast, please consider zipping

For a large arcyd installation, the var/log/debug log file can rotate completely multiple times per day. This means that it's not possible to look into events from previous days.

Please consider zipping the rotated logs so that we can afford to store more logs.

Arcyd: add reason for stopping to info logs

When Arcyd stops gracefully, a log entry is left in var/log/info. This is useful for knowing when Arcyd was stopped, it would also be useful to have a reason in the log too.

This could be supplied as an argument to arcyd stop and arcyd restart.

Make arcyon Windows compatible

When trying to run arcyon on a Windows environment, you get the error:

$ arcyon
Traceback (most recent call last):
  File "c:/Program Files/arcanist/phabricator-tools/bin/arcyon", line 31, in <module>
    import aoncmd_arcyon  # noqa
  File "c:\Program Files\arcanist\phabricator-tools\py\aon\aoncmd_arcyon.py", line 33, in <module>
    import aoncmd_gitdiffhelper
  File "c:\Program Files\arcanist\phabricator-tools\py\aon\aoncmd_gitdiffhelper.py", line 31, in <module>
    import abdt_branch
  File "c:\Program Files\arcanist\phabricator-tools\py\abd\abdt_branch.py", line 61, in <module>
    import abdt_differ
  File "c:\Program Files\arcanist\phabricator-tools\py\abd\abdt_differ.py", line 32, in <module>
    import phlgit_diff
  File "c:\Program Files\arcanist\phabricator-tools\py\phl\phlgit_diff.py", line 26, in <module>
    import phlsys_fs
  File "c:\Program Files\arcanist\phabricator-tools\py\phl\phlsys_fs.py", line 33, in <module>
    import fcntl
ImportError: No module named fcntl

It seems that fcntl is used to get file lock permissions, but no fcntl module exists for Windows. It's worth noting that arcanist uses the EDITOR config variable to get blocking sessions (PhutilInteractiveEditor.php) so it is possible, just not with the fcntl module.

Worth noting, I have no idea if this is the only code that's not Windows compatible, so this request may be more than just what I've listed here.

Arcyd: Increase the size of individual debug logs to 50 MB

This a follow-up to #24. When that PR is merged, the size of the logs would be reduced but the amount of logs available would still be the same. Increase the value of maxBytes for the file handler to be 50 MB which is currently 10 MB.

The (maximum) total size of the logs would be ~100 MB (50 + (50 * 10) / 10) assuming a gzip compression factor 10. On the other hand, current (maximum) total size of the logs is 110 MB (10 + (10 * 10)). So we'll be able to store 5 times more logs without actually increasing the total size of the logs stored.

REQUEST: Integration of vcs-user and Repository support.

I just used the puppet resources of this repo to setup a phabricator VM and debployment and usage worked really well.

But I somehow miss a comprehensive configuration of diffusion and the repo tool.
Creating a new repo and directly pushing to it is not possible directly.
I had to invest some time of extra configuration and doc reading to make repo hosting and integration work.
Is this something that will change in future ?

'vagrant up' failing --- Cannot understand environment setting "http_proxy="

I'm a complete vagrant n00b, so I'll do my best to provide the relevant info. For all I know, I just have a messed-up setup.

When following the tutorial for creating a new vm (cd vagrant && vagrant up), this happens mid-way through the setup:

warning: Exec[git clone http://github.com/facebook/arcanist /phabricator/instances/dev/arcanist](provider=posix): Cannot understand environment setting "http_proxy="
debug: Executing 'git clone http://github.com/facebook/arcanist /phabricator/instances/dev/arcanist'
err: /Stage[main]/Phabricator/Phabricator::Phabgitclone[arcanist]/Exec[git clone http://github.com/facebook/arcanist /phabricator/instances/dev/arcanist]/returns: change from notrun to 0 failed: Command exceeded timeout at /tmp/vagrant-puppet-1/manifests/phabricator/manifests/default.pp:83
debug: Exec[git clone http://github.com/facebook/phabricator /phabricator/instances/dev/phabricator](provider=posix): Executing 'git clone http://github.com/facebook/phabricator /phabricator/instances/dev/phabricator'
warning: Exec[git clone http://github.com/facebook/phabricator /phabricator/instances/dev/phabricator](provider=posix): Cannot understand environment setting "http_proxy="

If it's relevant, this is the output of vagrant box list. (the host system is OSX, btw):

$ vagrant box list
lucid32   (virtualbox)
precise32 (virtualbox)

Please let me know if you need any other information -- or if I'm just doing something wrong.

Thanks!

arcyon doesn't push linebreaks correctly into conduit

Thanks for the super-quick resolution of #9 . As a reward, here is another issue that I noticed.

If I run an arcyon command:

arcyon task-update --uri $URI --user $USER --cert $CERT --comment "This is a comment with a\nline break" --format-id $TASK_ID

Then I get a comment on the task that says:

This is a comment with a\nline break

Whereas, if I run this through Conduit directly (via the web interface), I get the intended result:

This is a comment with a
line break

Seems like arcyon does something along the way or uses some methods that breaks the text encoding somehow.

mysql user account issue when anonymous users are disabled

Usually I disable all anonymous users on a mysql installation for security reasons. I had done so with the installation where I tested this and ran in to issues with puppet default manifest which assumes you can connect to the mysql instance as an anonymous user.

We should probably list this as part of the README.md that users who have disabled anonymous access should add -u flag and --user grab as follows to the default.pp file

-    exec { "mysql < ${phab_dir}/initial.db && ${dev_dir}/phabricator/bin/storage upgrade --force":
+    exec { "mysql -u phab < ${phab_dir}/initial.db && ${dev_dir}/phabricator/bin/storage upgrade --user phab --force":

Update documentation and code to stay up to date with new phabricator version

Hi please update phabricator and documentation since phabricator upstream has been upgraded and your using a really outdated version.

Arcyd: Create a separate error log stream for errors

Currently, we only have info INFO and DEBUG logs for arcyd so any errors that are encountered also end in INFO logs. Ideally, we'd want to have a separate log for ERROR and the errors should not be in INFO logs at all.

Phabricator has removed Certificate page

See: https://secure.phabricator.com/w/changelog/2015.45/

"Removed obsolete "Conduit Certificate" page ... "

They have removed the Certificate page from Phabricator. Is there a way of using the newer API tokens with the phabricator-tools?

Arcyd: Make arcyd respect no_proxy settings

Currently arcyd takes http_proxy and https_proxy as argument and the values are passed by the environment variables. So if your proxy is set and you're using http://127.0.0.1 to access phabricator, the command errors out. This happens because it tries to access a local address via a proxy and ends up generating a '403 Access Denied'.

We should add a no_proxy parameter to the command and pass its value from environment variable, if set.

Meta: meta/autofix/fixlegal.py fails when invoked on new (uncommitted) files