subprocess.Popen in manager.py looking for directories not specified by user

[ericaltendorf]

subprocess.Popen in manager.py looking for directories not specified by user

[altendky]

It is trying to run the chia command to start a plot. I think the general expectation is that you have activated the chia virtual environment such that you could manually run chia.

plotman/manager.py

Lines 104 to 123 in 349d13a

	plot_args = ['chia', 'plots', 'create',
	'-k', str(plotting_cfg['k']),
	'-r', str(plotting_cfg['n_threads']),
	'-u', str(plotting_cfg['n_buckets']),
	'-b', str(plotting_cfg['job_buffer']),
	'-t', tmpdir,
	'-d', dstdir ]
	if 'e' in plotting_cfg and plotting_cfg['e']:
	plot_args.append('-e')
	if 'tmp2' in dir_cfg:
	plot_args.append('-2')
	plot_args.append(dir_cfg['tmp2'])
	logmsg = ('Starting plot job: %s ; logging to %s' % (' '.join(plot_args), logfile))
	# start_new_sessions to make the job independent of this controlling tty.
	p = subprocess.Popen(plot_args,
	stdout=open(logfile, 'w'),
	stderr=subprocess.STDOUT,
	start_new_session=True)

[altendky]

Perhaps it would be worth catching this particular exception and reporting an explanatory and instructional message.

[wbuchanan]

I thought I was calling it from a shell where I did have the virtual environment activated, but can double check later (I kicked off a job manually to start plotting). I definitely think more informative exception messages could be useful.

[wbuchanan]

I've run the commands from within the virtual environment and get the same errors.

[wbuchanan]

Calling from the virtual environment I get a new error:

...starting plot loop
Warning: unrecognized args: -k32 -n1
Warning: unrecognized args: -t/media/billy/Data/chiatmp -2/media/billy/Data/chiatmp
Warning: unrecognized args: -d/media/billy/chia-1-16TB/chiaplots -b13500
Warning: unrecognized args: -u64 -r4
Warning: unrecognized args: -a2792509996 -e
Traceback (most recent call last):
  File "plotman.py", line 92, in <module>
    wait_reason = manager.maybe_start_new_plot(dir_cfg, sched_cfg, plotting_cfg)
  File "/home/billy/Desktop/chia-blockchain/manager.py", line 69, in maybe_start_new_plot
    jobs = job.Job.get_running_jobs(dir_cfg['log'])
  File "/home/billy/Desktop/chia-blockchain/job.py", line 74, in get_running_jobs
    jobs.append(Job(proc, logroot))
  File "/home/billy/Desktop/chia-blockchain/job.py", line 127, in __init__
    self.init_from_logfile()
  File "/home/billy/Desktop/chia-blockchain/job.py", line 133, in init_from_logfile
    assert self.logfile
AssertionError

I had plotman cloned to ~/Desktop and navigated there and tried things, and got an error. Then I copied the plotman directory into the directory where I have the chia stuff located ~/Desktop/chia-blockchain/plotman, tried running the command there and got an error. Then I tried moving the individual python scripts into ~/Desktop/chia-blockchain and tried to execute the script there and the trackback above is from that last attempt.

I think in addition to having more informative exception messages, it might be useful to have a bit more information available about the installations/set-up assumptions as well.

[ericaltendorf]

Completely agreed. Volunteers are actively working on improving the installation/setup process and assumptions.

The other error you're seing (unrecognized args) is plotman trying to make sense of a plotting job that was started not-by-plotman with different format command line. This is something it can't quite do yet, although there are some fixes coming that should make it more robust to that.

[altendky]

(eric beat me to submitting this, but since i had already typed it up... and i totally agree with you as well...)

The present tendencies around setting up Chia and plotman do seem messy and confusing to me. Chia is already available on PyPI and we're working towards plotman being available as well. At that point users won't need to involve git in the setup at all. Hopefully then we can provide tidier setup instructions.

Probably best not to be moving files around. Their absolute location isn't super relevant and definitely has nothing to do with being 'in the venv'.

As to those warnings, you seem to be hitting the issue being worked on over in #60. plotman has assumed that arguments would have spaces such as -k 32 rather than -k32 as it is finding in situation. I think the fix is good but we're trying to coordinate a few changes. I'll try to chat with the other devs about that. If you want to try that PR you can work with the jkbecker-fix41b branch at https://github.com/jkbecker/plotman.

Sorry for all the troubles.

[altendky]

FYI, I merged the argument-without-space fix into development.

[wbuchanan]

@altendky
I figured that and tried modifying the code to add a space in the definition but it was still removed.

[wbuchanan]

@ericaltendorf
I'm not sure that was the cause of the error since I didn't have any other plotting happening at that time.

[wbuchanan]

@altendky
It seems like whatever changes you made in development worked. Do you know if the chia GUI is able to detect plotting that is started using plotman? At the moment it doesn't seem like that is happening so I can't tell how many actual plots are being created at the moment, but it seems like it is only a single plot? If that is correct, is there a way to force multiple plots to be created in parallel?

[altendky]

I have no idea what the GUI does, sorry. If you want to do a manual check of what is running to compare with the GUI and plotman, there's ps aux | grep 'chia plots create'. plotman can be configured in various ways with per temp drive limits, global limits, phase staggered starts, time staggered starts, and i think more. Take a look through the example configuration. Maybe start a new ticket or check in on keybase if you need more help with that.

Is the original issue reported here still outstanding? Or has it been resolved.

[wbuchanan]

@altendky
Sorry I should have closed the ticket. It's running now.

[sweetcake2017]

@altendky
Sorry I should have closed the ticket. It's running now.

hi ,i try to working on WSL2. The same problem heppen to me , can you tell me how to fix this proble? thx.

Traceback (most recent call last):
  File "/home/cake/.local/bin/plotman", line 8, in <module>
    sys.exit(main())
  File "/home/cake/.local/lib/python3.8/site-packages/plotman/plotman.py", line 145, in main
    wait_reason = manager.maybe_start_new_plot(cfg.directories, cfg.scheduling, cfg.plotting)
  File "/home/cake/.local/lib/python3.8/site-packages/plotman/manager.py", line 130, in maybe_start_new_plot
    p = subprocess.Popen(plot_args,
  File "/usr/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'chia'

[kalokola]

if you are using linux sudo apt-get install xclip