pidgeot/python-lnp

Cross-platform re-implementation of the Lazy Newb Pack launcher.

python-lnp's Issues

Delay finding web browser until needed

/u/ciphertext008 reports significant delays on startup, related to finding a web browser:

The [PyLNP] tries to check every directory in the system PATH variable for a web browser, I have a lot of stuff in my PATH. Perhaps this web browser check can be delayed until a user performs an action that would require knowing where the web browser is? (I have IE uninstalled)

Given that a web browser is not a core function - only needed when a user chooses to update the pack or visit a url from the links menu - could the search for it be delayed until it's required?

[Issue created by PeridexisErrant: 2014-09-19]

[Comment created by Pidgeot: 2014-09-19]
Delay import of webbrowser module to prevent performance issues on launch on Windows (fixes #26)

→ <>

[Comment created by Pidgeot: 2014-09-19]
Looking at the code, this is caused by the webbrowser module built-in to Python - it scans the path when the module is imported, and picks up on specific browsers to use as a fallback if the normal method of opening URLs on Windows doesn't work (which has the behavior /u/ciphertext008 wants).

While I can't control whether or not that happens, I can at least control when it happens. Fixed locally, will be pushed later...

Improve and distribute documentation

Beyond a certain point - which I think has recently been passed - a single README file is no longer sufficient for project documentation.

I propose to split the docs into three or four sections, each somewhat expanded:

• an introduction to the basics of PyLNP, for all users (especially end-users)
• cover PyLNP's content handling, aimed at content or package creators and maintainers
• building the application and platform-specific caveats
• developer information (mostly in the code)

Sphinx is the obvious tool for this, being the standard for Python documentation. Ideally we'd put the results online (at eg readthedocs.io) and also distribute the HTML with PyLNP (possibly baked into the executable with a local server so they can't get lost).

[Issue created by PeridexisErrant: 2016-08-24]
[Last updated on bitbucket: 2017-11-23]

[Comment created by PeridexisErrant: 2017-06-11]
No worries - that branch was really intended as a minimal proof of concept to build on. The 'readthedocs' theme is a lot nicer than 'alabaster', IMO, though it's also an extra package to install.

And it's been long enough that I have no idea what I was thinking with the code changes (and left a useless commit message), so just leave that out I think.

[Comment created by Pidgeot: 2017-06-10]
@PeridexisErrant I will most likely merge this fairly soon after the next release. That doesn't mean it'll be fully generated and distributed just yet - it needs some more tweaking (e.g. the theme does not work well in IE/Edge, and I'm not too keen on serif fonts for HTML) - but it's a step in the right direction.

I notice you have actual code modifications in mods.py - please confirm whether or not these need to be merged!

[Comment created by jecowa: 2016-10-20]
Utility manifests don't work with apps. I'd like to be able to keep using "Utilities.txt" for now. The only app I know that comes with a utility manifest is Armok Vision. I just have to rename the manifest.json to fix it.

[Comment created by PeridexisErrant: 2016-10-20]
@Pidgeot - there is a lot of documentation on the three old ways to configure utilities. Given that manifests are now widely used, and often distributed by utility authors, can we remove or at least deprecate the old strategies?

[Comment created by PeridexisErrant: 2016-10-20]
https://bitbucket.org/PeridexisErrant/python-lnp/branch/sphinx-docs

Basically done. I've essentially reorganised the existing README into logical files and edited the running-and-building section for flow. There were a few out-of-date things I cut, eg around DFHack init files (now run automatically).

I also added autogenerated API docs for everything; to get the most out of them we'll need to go through and improve the docstrings a lot. But that's a big job, and one that can be done incrementally...

Take a look, try it out, and if it seems good merge it! The only thing left is to somehow integrate it with the GUI and ensure it's distributed with the built version, but that's not my area.

[Comment created by Pidgeot: 2016-08-24]
I started looking into this a while back, but then other things got in the way...

I agree that Sphinx seems like the better choice, but I'm not yet 100% sure on the hosting part.

I would suggest that writing the documentation should be the first priority with this issue. Once we reach a decent enough starting point that it can be published, I believe we would also be much closer.

I've created a fork at https://bitbucket.org/Pidgeot/python-lnp_doctest, and included a stub Sphinx project. I'd suggest using that for any testing we'd need to do to make sure Sphinx really is the right tool for the job... then we can bring it in here and start writing it properly.

[Comment created by jecowa: 2016-08-31]
Those four slices look like a good way to divide up the manual. I don't know much about Sphinx, but I'm familiar with HTML.

[Comment created by Pidgeot: 2017-09-17]
I have finally grafted the documentation changes from that branch onto my local development branch. I expect to submit it in the next couple of days, once I decide how I want to handle hosting (it's either going to be self-hosting, or via readthedocs).

[Comment created by Pidgeot: 2017-11-23]
Change help option to point to online documentation
Point to online documentation in readme files (fixes #125)

→ <>

Linux: Window doesn't take correct minimum size

On Linux, it is possible to make the resizable window a tiny bit smaller than it should be, which makes it possible to hide the bottom row of buttons until the window is made larger again. This sould be fixed somehow.

[Issue created by Pidgeot: 2014-09-20]
[Last updated on bitbucket: 2015-03-05]

[Comment created by Pidgeot: 2015-03-05]
Fix window size on Linux (fixes #28)

→ <>

[Comment created by Pidgeot: 2015-02-05]
Closing as WONTFIX since there doesn't seem to be a way around this.

[Comment created by Pidgeot: 2014-11-08]
The problem seems to be that it is not consistent about what is being included and what isn't. Here's the basic outline of the relevant code

#!python

#Create all tabs, controls, etc.
root.update() # Process pending events to ensure everything has correct information
root.minsize(width=root.winfo_width(), height=root.winfo_height()) #Ask the window for its size and set that as the minimum 
#Start the main loop

As far as I can tell, the size being reported back in step 3 leaves out some part of the window, at least on my test VM - possibly the menu bar; I'm not entirely sure. However, when it is actually enforcing the size restrictions, it includes all of the window.

I have not yet been able to figure out a way around this; my best guess so far has been to handle the sizing portion after starting the main loop, but that had the same result. I suspect it might require delving deeper into the Tk code and figuring out how it measures the window size when enforcing those restrictions.

[Comment created by rx80: 2014-11-08]
I believe on linux (maybe depending on the display manager) the window decorations (border, title, etc.) are included in the window's size.

[Comment created by Pidgeot: 2014-11-08]
The height of the menubar is not even close to being measured correctly, it comes back as approx. 100 pixels (!) on both Windows and Linux, even if checked during resizing. I'm not sure how it arrives at that figure, but it forces the window much larger than necessary, and it still allows you to make it slightly smaller than that, since minsize (naturally) still isn't behaving correctly (since it apparently uses a different way of measuring).

[Comment created by rx80: 2014-11-08]
This seems more appropriate, again tested on ubuntu and windows: https://bitbucket.org/rx80/python-lnp/commits/294ff4f014bdcca701dc8fb67b9f9dff764e0c67

[Comment created by Pidgeot: 2014-11-08]
Did not fix it on my test machine (Mint 15/MATE). Feel free to propose any other ideas if you come up with any, though.

[Comment created by rx80: 2014-11-08]
This looks like it fixes it on ubuntu, though it doesn't seem perfect or reasonable:
https://bitbucket.org/rx80/python-lnp/commits/55fdcdde0972a85c483550276d8638d6b716394d

[Comment created by rx80: 2014-11-08]
After a bit of testing... it's the menu bar that's somehow not calculated into the height. If you comment out the line

#!python

self.create_menu(root)

everything works as expected... the height of the menubar is reported as "1"

[Comment created by Pidgeot: 2014-11-08]
Already did: I asked Tk to run the sizing related code once the main loop was idle (using root.after_idle), and the issue persisted.

[Comment created by rx80: 2014-11-08]
Oh, i believe that is another issue. Try and request the sizes, and set the min size, only after the window has been painted once.

Toggle for 'pause on job cancellation'

Thorfinn asks on a bay12 forums thread:

Quick suggestion. I'd appreciate a clickbox to enable/disable pause on job cancellation, mostly for use on aquifer and other early game stuff. Yes, it's explained right at the top of the file, but I'm finding myself answering various permutations of this question from my less-savvy friends a couple times a week, at least. "Which file am I supposed to change?", "Which line?", "What do I change it to again?" and inevitably, "How do I get rid of all these maddening pauses?"

I'm thinking on the options tab with the rest of the gameplay options would work swimmingly.

[Issue created by PeridexisErrant: 2017-02-19]
[Last updated on bitbucket: 2017-11-23]

[Comment created by Pidgeot: 2017-05-06]
This is not in the init file, but in the announcements. These are the announcements that will cause the game to pause (as of 43.05):

[ENDGAME_EVENT_1:A_D:D_D:BOX:P:R]
[ENDGAME_EVENT_2:A_D:D_D:BOX:P:R]
[FEATURE_DISCOVERY:A_D:D_D:BOX:P:R]
[STRUCK_DEEP_METAL:A_D:D_D:BOX:P:R]
[MIGRANT_ARRIVAL_NAMED:A_D:D_D:P:R]
[MIGRANT_ARRIVAL:A_D:D_D:P:R]
[DIG_CANCEL_WARM:A_D:D_D:P:R]
[DIG_CANCEL_DAMP:A_D:D_D:P:R]
[AMBUSH_DEFENDER:A_D:D_D:P:R]
[AMBUSH_RESIDENT:A_D:D_D:P:R]
[AMBUSH_THIEF:A_D:D_D:P:R]
[AMBUSH_THIEF_SUPPORT_SKULKING:A_D:D_D:P:R]
[AMBUSH_THIEF_SUPPORT_NATURE:A_D:D_D:P:R]
[AMBUSH_THIEF_SUPPORT:A_D:D_D:P:R]
[AMBUSH_MISCHIEVOUS:A_D:D_D:P:R]
[AMBUSH_SNATCHER:A_D:D_D:P:R]
[AMBUSH_SNATCHER_SUPPORT:A_D:D_D:P:R]
[AMBUSH_AMBUSHER_NATURE:A_D:D_D:P:R]
[AMBUSH_AMBUSHER:A_D:D_D:P:R]
[AMBUSH_INJURED:A_D:D_D:P:R]
[AMBUSH_OTHER:A_D:D_D:P:R]
[AMBUSH_INCAPACITATED:A_D:D_D:P:R]
[CARAVAN_ARRIVAL:A_D:D_D:P:R]
[NOBLE_ARRIVAL:A_D:D_D:P:R]
[D_MIGRANTS_ARRIVAL:A_D:D_D:P:R]
[D_MIGRANT_ARRIVAL:A_D:D_D:P:R]
[ANIMAL_TRAP_CATCH:A_D:D_D:P:R]
[ANIMAL_TRAP_ROBBED:A_D:D_D:P:R]
[MISCHIEF_LEVER:A_D:D_D:P:R]
[MISCHIEF_PLATE:A_D:D_D:P:R]
[MISCHIEF_CAGE:A_D:D_D:P:R]
[MISCHIEF_CHAIN:A_D:D_D:P:R]
[DIPLOMAT_ARRIVAL:A_D:D_D:P:R]
[LIAISON_ARRIVAL:A_D:D_D:P:R]
[TRADE_DIPLOMAT_ARRIVAL:A_D:D_D:P:R]
[CAVE_COLLAPSE:A_D:D_D:P:R]
[BIRTH_CITIZEN:A_D:D_D:P:R]
[STRANGE_MOOD:A_D:D_D:P:R]
[MADE_ARTIFACT:A_D:D_D:BOX:P:R]
[NAMED_ARTIFACT:A_D:D_D:BOX:P:R]
[VERMIN_CAGE_ESCAPE:A_D:D_D:P:R]
[TRIGGER_WEB:A_D:D_D:P:R]
[MOOD_BUILDING_CLAIMED:A_D:D_D:P:R]
[ARTIFACT_BEGUN:A_D:D_D:P:R]
[MEGABEAST_ARRIVAL:A_D:D_D:BOX:P:R]
[BERSERK_CITIZEN:A_D:D_D:P:R]
[UNDEAD_ATTACK:A_D:D_D:BOX:P:R]
[STRANGE_CLOUD:A_D:D_D:P:R]
[TRAINING_FULL_REVERSION:A_D:D_D:P:R]
[FORT_POSITION_SUCCESSION:A_D:D_D:BOX:P:R]
[DEITY_CURSE:A_D:D_D:BOX:P:R]

I assume this is a request to easily disable DIG_CANCEL_DAMP (maybe also DIG_CANCEL_WARM)? We could do that as a standalone option, but if there's more it would be better to write an announcements editor...

[Comment created by Pidgeot: 2017-09-17]
I've just implemented options specifically for damp and warm stone; turning them off removes the P and R flags, while turning them on adds both of those flags back in.

[Comment created by Pidgeot: 2017-11-23]
Implement pause toggle for damp and warm stone announcements (fixes #136)

→ <<cset 585096306b19>>

Graphics simplification was broken by pull #10

Sorry!

It's been fixed here @da055a4, by which stage the branch had also added handling for data/speech (closing #49).

I'd do a pull request, but development in the branch is ongoing so it's probably better for you to merge from your end.

[Issue created by PeridexisErrant: 2015-01-12]

[Comment created by Pidgeot: 2015-01-12]
Closed by pull request #18.

[Comment created by Pidgeot: 2015-01-12]
This may be because it is almost 2:30 AM here, but it is not immediately obvious to me exactly how it is currently broken. As far as I can tell, the commit you refer to should copy the same files as the current code (excluding data/speech, that is).

[Comment created by PeridexisErrant: 2015-01-12]
It should - but it was getting rid of the init files. The new version fixed that by changing the glob section and explicitly handling init files.

updating graphics fails for "fresh" install

This might only affect cases where there is no existing settings file to read.

#!text
Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1489, in __call__
    return self.func(*args)
  File "/.../tkgui.py", line 749, in <lambda>
    command=lambda: self.install_graphics(graphicpacks))
  File "/.../tkgui.py", line 1497, in install_graphics
    self.load_params()
  File "/.../tkgui.py", line 1160, in load_params
    self.lnp.load_params()
  File "./lnp.py", line 109, in load_params
    self.settings.read_settings()
  File "/.../settings.py", line 186, in read_settings
    self.read_file(filename, self.in_files[files], len(files) == 1)
  File "/.../settings.py", line 225, in read_file
    self.settings[field] = match.group(1)
AttributeError: 'NoneType' object has no attribute 'group'

Recommend additional guard against regex match failing; attached patch is against tip.

[Issue created by beltorak: 2014-09-01]
[Last updated on bitbucket: 2014-09-02]

[Comment created by Pidgeot: 2014-09-02]
Avoid crashing if an expected field cannot be found, print warning instead (fixes #21)

→ <>

[Comment created by beltorak: 2014-09-02]
You are right; the obsidian pack is just not updated for DF2014 (0.40); The wiki needs updating. I found the linux LNP at http://www.bay12forums.com/smf/index.php?topic=140966 which led me to an updated collection of graphics packs at https://github.com/fricy/DFgraphics. I'll stick to those.

This issue can be closed or rejected.

Thanks

[Comment created by beltorak: 2014-09-01]
It's the obsidian graphics set, and it only happens when I try to save. After launching a few times the problem went away, which leads me to think that it only happens when it can't find the setting.

Sorry about the noise on the issue tracker, i realized i made the guard bypass the force_bool check; but you seem to have it now.

Graphics pack is from http://dffd.wimbli.com/download.php?id=7728&f=%5B16x16%5D+Obsidian+%28v.0.8%29.zip

Thanks

[Comment created by Pidgeot: 2014-09-01]
What version of DF were you trying to install to? The pack seems to be for 0.34; if you installed it over a 0.40 DF, that explains what went wrong. The program expects to find all the tags it originally saw in both d_init.txt and init.txt; if you installed a pack for an older version, that explains the break. (It still needs to be fixed, but in that case I should probably throw a warning.)

[Comment created by Pidgeot: 2014-09-01]
Going by the stack trace, this happened after a graphics pack install - or at the very least after some modification of some files, because otherwise, this problem would have shown itself on launch. The pack you installed may be broken (or at least missing something that I expect it to have).

I'm putting in a fix similar to your patch now, but before pushing it online, I'd like to see the pack itself to figure out why it's breaking in the first place - a link would be much appreciated.

[Comment created by beltorak: 2014-09-01]
that could be it; let me see what happens with phoebus.

Cannot launch Dwarf Fortress if it's installed in a directory with spaces

• Ubuntu 16.04
• Lazy Newb Pack 0.44.09-r01
• PyLNP 0.13

My Dwarf fortress is mounted on /media/[user]/Slightly Darker Black

When attempting to launch with either terminal, I get the error

'Failed to execute child process "/media/[user]/Slightly" (No such file or directory)'

The stderr.txt appears not releavant, but here it is regardless. Note, my username is not "michael":

#!

Running PyLNP 0.13 (OS: linux, Compiled: True)
INFO: Read installed graphics (Mayday) from log 
Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "/home/michael/Desktop/df/python_lnp/build/lnp/out00-PYZ.pyz/core.lnp", line 126, in __init__
  File "/home/michael/Desktop/df/python_lnp/build/lnp/out00-PYZ.pyz/tkgui.tkgui", line 251, in start
  File "/home/michael/Desktop/df/python_lnp/build/lnp/out00-PYZ.pyz/Tkinter", line 1121, in mainloop
  File "/home/michael/Desktop/df/python_lnp/build/lnp/out00-PYZ.pyz/Tkinter", line 1527, in __call__
KeyboardInterrupt

[Issue created by Former user: 2018-04-24]
[Last updated on bitbucket: 2018-05-06]

[Comment created by Pidgeot: 2018-04-24]
Just for reference which terminals are you attempting? The list of available terminals depends on what PyLNP finds on your system.

It's likely independent of that choice, but I just want to be sure.

[Comment created by Pidgeot: 2018-05-06]
I am not able to reproduce this, neither when running PyLNP from source or when using the pre-built executable. This was when running everything from a folder ~/Desktop/Dwarf Fortress/df

I believe the pack includes a shell script to launch everything: it may be this one that's having a bad side effect. Try running the PyLNP executable directly and see if you get a different result; if so, please report this to the pack maintainer instead.

If you still have problems in either case, try launching PyLNP with the -d parameter and then provide the contents of both stderr.txt and stdout.txt once you've reproduced the problem.

Python 3 unicode/bytes issue with KDE terminal

I tried to launch pylnp from source-- that is, using launch.py rather than the compiled binary-- with Python 3 on Fedora. It worked until I actually tried to launch DF, at which point I got this traceback:

‌

Traceback (most recent call last):
  File "/home/bjr/.pylnp/pylnp/tkgui/tkgui.py", line 542, in run_df
    launcher.run_df()
  File "/home/bjr/.pylnp/pylnp/core/launcher.py", line 52, in run_df
    result = run_program(executable, force, True, spawn_terminal)
  File "/home/bjr/.pylnp/pylnp/core/launcher.py", line 91, in run_program
    run_args = terminal.get_terminal_command([path,])
  File "/home/bjr/.pylnp/pylnp/core/terminal.py", line 25, in get_terminal_command
    term = get_configured_terminal().get_command_line()
  File "/home/bjr/.pylnp/pylnp/core/terminal.py", line 125, in get_command_line
    '\n', '')
TypeError: a bytes-like object is required, not 'str'

The KDETerminal implementation of get_command_line looks like this:

‌

    @staticmethod
    def get_command_line():
        s = subprocess.check_output(
            ['kreadconfig', '--file', 'kdeglobals', '--group', 'General',
             '--key', 'TerminalApplication', '--default', 'konsole']).replace(
                 '\n', '')
        return ['nohup', s, '-e']

I was able to fix this by just changing the arguments to .replace() to be binary strings (e.g. “b'\n' and b''). Not sure if there’s a more elegant thing to do here. And looking at some of the other code in this module, they seem to have similar looking code, which suggests to me other places might have this problem too.

‌

[Issue created by TC01: 2019-09-15]
[Last updated on bitbucket: 2020-04-04]

[Comment created by Pidgeot: 2019-09-15]
I’ll have to verify it doesn’t break other Python versions, but otherwise it seems like a reasonable enough change.

Just for reference, can you check exactly which version of Python you’re running?

[Comment created by TC01: 2019-09-15]
Sure! I’m using Python 3.7.4 here.

[Comment created by McArcady: 2020-04-04]
This problem may be fixed by the github PR#3 #3, that fixes some terminal handling issues with python3 on Linux.

I’ve not tested it with the KDE Terminal though.

[Comment created by Pidgeot: 2020-04-04]
I’ve merged the pull request over on Github, it’ll appear on Bitbucket shortly.

I'm in the process of migrating over the Bitbucket issues now, so please open a new issue over on Github if this happens again.

New release request

Now that some improvements for macOS have been merged in, could we tag a suitable version 0.14 and make some release artifacts? This would make it easier to release the next Mac starter pack.

If this isn't possible in the near future, I could just use a specific revision and build it myself, but it would be preferred to stick to your releases.

I can probably build and sign it and upload the binary here, if that's preferable, but since I think PyInstaller copies the system frameworks into the bundle, it will only work on 10.13 and newer.

[Issue created by rgov: 2018-07-18]
[Last updated on bitbucket: 2018-07-20]

[Comment created by Pidgeot: 2018-07-18]
I'm aiming to get a release out soon, within a week.

There's one or two things I want to check up on first, so can't give an exact date, but it's coming.

[Comment created by Pidgeot: 2018-07-19]
New version is out now, although it is not 0.14 but 0.13a :)

[Comment created by Pidgeot: 2018-07-19]
Removing milestone: 0.13a (automated comment)

[Comment created by rgov: 2018-07-20]
Thanks! I've confirmed that it works on 10.13.5 and retina graphics are now working.

DFHack freezes PyLNP

Thought I'd open an issue here instead of doing it over in the thread again.

Issue is as the title says. Not sure if this is something you can fix.
To clarify the issue just in case. If I set Dwarf Therapist to auto start with DF, it doesn't, until I terminate DFHack, upon which Dwarf Therapist loads. Clicking on the frozen window even gets the action through after it unfreezes.

For me the issue itself is trival, I just launch the utilities I want to use myself now.

Here's a trace from sending an interrupt to the 'python launch.py' process. Note I give two interrupts. The first just terminates the DFHack process, and is probably the only relevant one. At this point the window unfreezes and I send another interrupt which terminates the PyLNP window after a while.

#!python

$> python2.7 launch.py
^CException in Tkinter callback
Traceback (most recent call last):
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1486, in __call__
    return self.func(*args)
  File "core/launcher.py", line 48, in run_df
    result = run_program(executable, force, True, spawn_terminal)
  File "core/launcher.py", line 96, in run_program
    retcode = subprocess.call(term + [path], cwd=workdir)
  File "/usr/lib/python2.7/subprocess.py", line 522, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 1376, in wait
    pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
  File "/usr/lib/python2.7/subprocess.py", line 476, in _eintr_retry_call
    return func(*args)
KeyboardInterrupt

^CTraceback (most recent call last):
  File "launch.py", line 11, in <module>
    lnp.PyLNP()
  File "core/lnp.py", line 77, in __init__
    self.ui.start()
  File "tkgui/tkgui.py", line 188, in start
    self.root.mainloop()
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1070, in mainloop
    self.tk.mainloop(n)
KeyboardInterrupt

[Issue created by AlexTes: 2014-11-20]

[Comment created by Pidgeot: 2014-11-20]
Add nohup to all xdg-terminal cases (fixes #45)

→ <<cset 206147a76875>>

[Comment created by AlexTes: 2014-11-20]
Silly Silly me. You fixed it Michael!

I noticed PermissionError: [Errno 13] Permission denied, tried running as super, which is noted to be a bad idea in the xdg-terminal script, didn't work and gave up.

However I was confused reading your comment because I didn't understand why you couldn't spawn an independent terminal. After figuring out your one way was of course the xdg-terminal script your adjustment of xdg-terminal broke. I tried to run it myself, separate, and see what the output was. This is when I discovered it wasn't you who broke it but me. The new script wasn't set executable, so after adding executable permission everything is working nicely.

Thanks Michael!

[Comment created by Pidgeot: 2014-11-20]
So just to be 100% clear: The nohup trick in the attached file worked? PyLNP no longer freezes, and quitting it while DFHack is running doesn't break anything?

[Comment created by AlexTes: 2014-11-20]
I'm using bspwm, which is a window manager (tiled) no DE. It gives me quite a bit of trouble with java applications as well, I might just switch in the future :/

Output: Desktop environment is generic.

(Also, damn you are fast)

[Comment created by AlexTes: 2014-11-20]
Yes, Yes and Yes ^^

[Comment created by Pidgeot: 2014-11-20]
What this means is that the xdg-terminal script couldn't launch a properly independent terminal - this is imperative for DFHack to work properly. This can be fixed (and I may know how), but I'll need to know something about your system, and you'll have to test the changes once I make them.

First of all, I will need to know which desktop environment and/or window manager you're using. It will also be helpful if you could check what the xdg-terminal script detects; try adding the line echo Desktop environment is $DE after detectDE near the end of the file and tell me what it prints.

[Comment created by AlexTes: 2014-11-20]
No more df I'm afraid.

#!python
Traceback (most recent call last):
  File "./core/launcher.py", line 96, in run_program
    retcode = subprocess.call(term + [path], cwd=workdir)
  File "/usr/lib/python3.4/subprocess.py", line 537, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/usr/lib/python3.4/subprocess.py", line 858, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.4/subprocess.py", line 1456, in _execute_child
    raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission denied
Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/lib/python3.4/tkinter/__init__.py", line 1533, in __call__
    return self.func(*args)
  File "./core/launcher.py", line 50, in run_df
    raise Exception('Failed to run Dwarf Fortress.')
Exception: Failed to run Dwarf Fortress.

Export returns XDG_SESSION_DESKTOP=bspwm

I do not understand the way bspwm actually works. As is mentioned in the configuration section bspwm receives information from X events and a socket, neither of which I know my way around. I spawn processes through sxhkd (which is also mentioned in the configuration section) or through dmenu. How these are then actually spawned I'm unsure.

Using xterm, a separate dfhack window spawns just fine with the original xdg-terminal file though! So this seems to be working fine. Perhaps you meant that it seems to be spawned as a child of the terminal I run the launcher from as dfhack terminates upon me giving an interrupt to the terminal that launched the LNP. Again, not sure what the issue is. Unfortunately my understanding of bspwm is small, although I love using it and have all I need with its config file.

bspwm users are rare, more rare than Arch users, and I have yet to meet one of even the latter, and that's as a Computer Science major. I wouldn't say this has priority if a proper DE / WM fixes the issue. However I don't mind trying my best at extending the script when I have some time.

[Comment created by AlexTes: 2014-11-20]
Whoops, completely forgot I read about this similar seeming issue where you adviced to flip two lines now in launcher.py, ps.wait and a ps.stdout.read. Shouldn't effect anything, didn't change the trace at a glance, but I'd rather not chance wasting your time ;]

[Comment created by Pidgeot: 2014-11-20]
As a quick test, try using the attached xdg-terminal script and launch DFHack through PyLNP. The expected result is:

• PyLNP does not freeze
• DFHack opens with a new terminal
• While DFHack is running, you should be able to close PyLNP without seeing any effects on DFHack

If this does not work, then the script needs to be extended with a way to detect bspwm and make it do this:

• For detecting bspwm: Look at the output of export to see if anything seems related to bspwm. If you see something, I'll need to know the name and value of the variables in question.
• For launchíng: I've browsed through the bspwm documentation, but I did not see a command to spawn a new terminal. It is entirely possible I have missed it; if you know something I don't, please let me know.

[Comment created by Pidgeot: 2014-11-20]
I'm making the Linux builds on Gentoo VMs now, and I have an Arch installation running on my Raspberry Pi - is that close enough? :P

Unfortunately, there's not much I can do without a way to spawn an independent terminal. This is a problem all launchers would experience.

Based on the sample sxhkdrc file, it seems like all it does is run urxvt - but there doesn't seem to be a way to actually obtain that information from a script (urxvt is merely the default; it shouldn't be required - and the hotkey I'm looking for doesn't have to be super + return).

You may have luck by modifying the terminal_generic function in xdg-terminal to explicitly run uxrvt (the nohup stuff from the previously attached file may or may not be necessary; I'm not sure), but without a foolproof way of handling bspwm, I can't really add a case for it to the "main" script. The guy who makes it may have an idea, but in that case, I'd prefer it if you asked him (you're the one using bspwm, so it's far more likely you can write the question in a proper way).

hide TWBT-Next png files from user

TWBT added a new feature (TWBT-Next / multi-layer rendering) that requires some additional graphics files, but these graphics file should not be visible / selectable from the "Customize" sub-tab of the "Graphics" tab of the PyLNP launcher.

These two files:

• transparent1px.png
• white1px.png

And also any file ending with -bg.png or -top.png.

[Issue created by jecowa: 2017-12-11]
[Last updated on bitbucket: 2018-01-06]

[Comment created by PeridexisErrant: 2017-12-25]
For the multi-layer files: we should only hide the layer files (bg and top) when both of them and the base file all exist. Otherwise +1 :-)

[Comment created by Pidgeot: 2018-01-06]
Done including @PeridexisErrant's suggestion.

[Comment created by Pidgeot: 2018-01-06]
Hide TWBT-Next files from tileset selection (fixes #151)

→ <<cset 8a3862d1668b>>

Pylnp 0.5 becomes unresponsive when launching dfhack and utilities - OSX 10.7.5

As the title says: when launching dfhack or a utility the Pylnp app stops responding. The logs are empty.
If there's no dfhack present, only vanilla df the launcher can start the game. Starting Therapist will freeze it nevertheless. Curiously if I change the contents of the "df" script with the contents of the "dfhack" script it can launch without freezing, but the game crashes immediately...
Further testing:
1, Launching native apps and dfhack locks up.
2, Launching .jar files works ok.
3, launching .sh script (soundsense) doesn't work - as in nothing happens, no lock up.

OS crash log is here

[Issue created by fricy: 2014-09-20]

[Comment created by fricy: 2014-09-20]
yes, this is ok.

[Comment created by Pidgeot: 2014-09-20]
I've no idea what went wrong, but I'll go replace the file under Downloads with this one, then.

[Comment created by Pidgeot: 2014-09-20]
Just to be completely certain about that, I did a rebuild without that debug code and attached it to this post.

Could you please confirm if it's still present?

[Comment created by fricy: 2014-09-20]
Debug build works ok, nothing interesting in the logs.

Running ps
Reading output
Waiting for process to terminate
Checking result
0

[Comment created by Pidgeot: 2014-09-20]
The only difference between this one and regular 0.5.1 is a few extra print statements when it searches for running processes. If the debug build works (including when run as a normal .app), but regular 0.5.1 doesn't, then there's nothing I can do.

Are you sure you were running 0.5.1 when you saw the hang that made you re-open? Maybe try deleting the old .app before extracting the new one, just in case?

[Comment created by Pidgeot: 2014-09-20]
...that stack trace makes no sense for this problem. Are you sure the program was hanging? Because that would indicate the UI had somehow gotten corrupted...

I've attached a build with some extra debug output, which will hopefully narrow this down a bit. Run it directly as before and check stderr.txt after you kill it.

[Comment created by fricy: 2014-09-20]
Checked it again to make sure: yepp. 0.5.1 hangs, debug 5.1 works. I suppose something went wrong with the building first time.

[Comment created by fricy: 2014-09-20]
Traceback (most recent call last):
File "", line 967, in
File "", line 91, in init
File "/Users/michael/Desktop/df/python_lnp/build/lnp/out00-PYZ.pyz/tkgui.tkgui", line 187, in start
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 1017, in mainloop
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 1414, in call
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 1175, in _report_exception
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 1108, in _root
KeyboardInterrupt

[Comment created by Pidgeot: 2014-09-20]
Try running the compiled app directly from a terminal - go into the Contents/MacOS folder and run ./PyLNP directly, then press Ctrl+C when it hangs. stderr.txt should have a Python stack trace; I'd like to know whether or not it's any different.

[Comment created by fricy: 2014-09-20]
Sorry, I have to reopen this. The source works correctly, but the compiled app doesn't, still freezes the same way.

[Comment created by Pidgeot: 2014-09-20]
Fix for OS X 10.7 issue (fixes #29)

→ <>

[Comment created by fricy: 2014-09-20]
Ok, cool. Thx for the soundsense fix and the support!

[Comment created by Pidgeot: 2014-09-20]
I'll make a V0.5.1 with that change and put it up shortly.

Regarding Soundsense: I tried downloading your MacNewbie pack, and just trying to launch that shell script directly gives me this error message:

-bash: ./soundSense.sh: /bin/sh^M: bad interpreter: no such file or directory

That tells me the shell script has Windows line endings, and indeed, changing the file to use Unix line endings makes the program launch successfully.

[Comment created by fricy: 2014-09-20]
1, ps axww is ok.
2, switching the lines solves the dfhack and .app launch problems.

+1 issue: back to my earlier comment about soundsense: There's a problem with launching shell scripts.
I relaized it's not soundsense.sh that is used to launch it, but a unix executable file in the same directory without extension. The problem is: it's not being detected by the launcher. When I create a shell script (that's picked up by pylnp) with the content ./SoundSense I get this error:

File "lnp.py", line 202, in run_program
self.running[path] = subprocess.Popen(run_args, cwd=workdir)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 672, in init
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1202, in _execute_child
OSError: [Errno 2] No such file or directory

[Comment created by Pidgeot: 2014-09-20]
Okay, at least now I see what's going on.

The issue comes from the newly added check to detect if a program is already running before re-launching it. For certain kinds of processes, I can maintain a reference to their process ID, but for OS X .app bundles and DFHack, I have to parse the output from the ps command. Apparently, it is waiting for that process to end.

I need you to check two things on your 10.7 machine:

1. Open a terminal and type ps axww, and tell me if that hangs.
2. Please try to change lnp.py by switching line 224 and 225 around (ps.wait() and s = ps.stdout.read()), then see if it still hangs.

[Comment created by fricy: 2014-09-20]
Unable to reproduce on 10.9.

[Comment created by fricy: 2014-09-20]
I managed to get a partial log from the app, it's generated after trying to launch soundsense.sh:

Traceback (most recent call last):
File "", line 202, in run_program
File "/Users/michael/Desktop/df/python_lnp/build/lnp/out00-PYZ.pyz/subprocess", line 679, in init
File "/Users/michael/Desktop/df/python_lnp/build/lnp/out00-PYZ.pyz/subprocess", line 1228, in _execute_child
OSError: [Errno 2] No such file or directory
0

Will test with 10.9 in a minute. The freeze happens with source too:

Frigyes-MacBook:Macnewbie fricy$ python lnp.py
Note: PIL not found and Tk version too old for PNG support (8.5).Falling back to GIF images.
^CException in Tkinter callback
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 1410, in call
File "lnp.py", line 152, in run_df
os.path.join(self.df_dir, 'dfhack'), force, True, True)
File "lnp.py", line 201, in run_program
if force or self.check_program_not_running(path, nonchild):
File "lnp.py", line 224, in check_program_not_running
ps.wait()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1243, in wait
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 471, in _eintr_retry_call
KeyboardInterrupt

[Comment created by Pidgeot: 2014-09-20]
I am unable to reproduce this on my machine (10.8).

Can you please check if the problem occurs when running from source code? If it does, and you launch it from a terminal, you should be able to use CTRL-C to force a Python stacktrace, which would give me a better chance of figuring out what is wrong.

OSX crash

0.3 runs ok on 10.9.3, crashes on 10.7.5.
Maybe related: I've seen a somewhat similar crash when making the app with tkinter framework (py2app), Pillow looks to work on both.

#!

Process:         launchd [14768]
Path:            /Users/USER/*/PyLNP.app/Contents/MacOS/PyLNP
Identifier:      org.pythonmac.unspecified.PyLNP
Version:         ??? (???)
Code Type:       X86-64 (Native)
Parent Process:  launchd [916]

Date/Time:       2014-08-31 23:20:27.923 +0200
OS Version:      Mac OS X 10.7.5 (11G63b)
Report Version:  9

Interval Since Last Report:          8660546 sec
Crashes Since Last Report:           1283
Per-App Crashes Since Last Report:   14
Anonymous UUID:                      22C45EF3-E6EB-42A1-BCCD-908F62176807

Crashed Thread:  Unknown

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x00007fff5fc01028

Backtrace not available

Unknown thread crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000055  rbx: 0x0000000000000000  rcx: 0x0000000000000000  rdx: 0x0000000000000000
  rdi: 0x0000000000000000  rsi: 0x0000000000000000  rbp: 0x0000000000000000  rsp: 0x0000000000000000
   r8: 0x0000000000000000   r9: 0x0000000000000000  r10: 0x0000000000000000  r11: 0x0000000000000000
  r12: 0x0000000000000000  r13: 0x0000000000000000  r14: 0x0000000000000000  r15: 0x0000000000000000
  rip: 0x00007fff5fc01028  rfl: 0x0000000000010203  cr2: 0x00007fff5fc01028
Logical CPU: 0

Binary images description not available


External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 1
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 9794
    thread_create: 0
    thread_set_state: 0

Model: MacBook5,2, BootROM MB52.0088.B06, 2 processors, Intel Core 2 Duo, 2.13 GHz, 4 GB, SMC 1.38f5
Graphics: NVIDIA GeForce 9400M, NVIDIA GeForce 9400M, PCI, 256 MB
Memory Module: BANK 0/DIMM0, 2 GB, DDR2 SDRAM, 800 MHz, 0x7F98000000000000, 0x202020202020202020202020202020202020
Memory Module: BANK 1/DIMM0, 2 GB, DDR2 SDRAM, 800 MHz, 0x7F98000000000000, 0x202020202020202020202020202020202020
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x8E), Broadcom BCM43xx 1.0 (5.106.198.19.22)
Bluetooth: Version 4.0.8f17, 2 service, 11 devices, 1 incoming serial ports
Network Service: Wi-Fi, AirPort, en1
Serial ATA Device: WDC WD5000BEKT-00KA9T0, 500,11 GB
Serial ATA Device: MATSHITADVD-R   UJ867A
USB Device: Built-in iSight, apple_vendor_id, 0x8501, 0x24400000 / 3
USB Device: Apple Internal Keyboard / Trackpad, apple_vendor_id, 0x022a, 0x04600000 / 3
USB Device: IR Receiver, apple_vendor_id, 0x8242, 0x04500000 / 2
USB Device: BRCM2046 Hub, 0x0a5c  (Broadcom Corp.), 0x4500, 0x06100000 / 2
USB Device: Bluetooth USB Host Controller, apple_vendor_id, 0x8215, 0x06110000 / 3

[Issue created by fricy: 2014-08-31]
[Last updated on bitbucket: 2014-09-02]

[Comment created by Pidgeot: 2014-09-02]
Change OS X builds to use PyInstaller as well (fixes #16)

→ <>

[Comment created by Pidgeot: 2014-09-02]
The problem with the missing image is strange, but please make a new ticket for that.

I'm going to switch OS X to PyInstaller starting with the next build, so this particular ticket should be fixed (once I commit and push the necessary changes).

[Comment created by fricy: 2014-09-02]
Tested on 10.7.5, clean-10.9.3 (factory conditions), and build-machine-10.9.3 (brewed python 2.7 with Pillow, no tkinter framework), loads on all systems.

However I just noticed another problem: The included graphics "header" (LNPSMALL) only displays on my 10.7.5 system, but not on 10.9.3.
I tested the official 0.3 version, this build, and my build own build too (homebrewed python 2.7 with Pillow, no tkinter framework, built with py2app), but none of them show the header on any of my 10.9.3 installs.
I'm a bit suprised that I haven't noticed it so far, sorry.

[Comment created by Pidgeot: 2014-09-02]
Can you please running try the attached build? This one is using PyInstaller instead of py2app; I'm hoping it might make a difference.

[Comment created by fricy: 2014-09-02]
http://pastebin.com/3Sz1jc4d
Line 4 was suspicious, but fixing the path by removing the excessive ../MacOS with install_name_tool didn't fix it. Can't spot anything obvious.

[Comment created by Pidgeot: 2014-09-02]
Given that I have no way of accessing a 10.7 machine, I have no way of reproducing this, and instead need to make some guesses based on what little information I can dig up about this...

As far as I can tell, this is possibly related to the program looking for something it can't find. However, if I don't know what part is failing, I can't figure out how to fix it.

I've made and attached a script which will give me some more information, and hopefully bring me a little closer to tracking down the problem. Can you please put the script in the same folder as PyLNP.app on your 10.7 Mac and attach the output from running it? (Save it, open a terminal, go to the folder, chmod +x deps.sh && ./deps.sh > deps.txt, then attach deps.txt here)

[Comment created by fricy: 2014-08-31]
Not really, just hoping I know what to poke. :)
I prefer the error logs in /Resources, in fact everything there. #18
elif sys.platform == 'darwin':
self.bundle = 'osx'
# OS X bundles start in different directory
os.chdir('../Resources')

[Comment created by Pidgeot: 2014-08-31]
That is where they currently are (which is actually a bug - they're supposed to go to the same folder as PyLNP.app, but clearly I'm not redirecting properly for the bundle).

Building against a different deployment target might work, but I'm not sure how exactly to go about that - I'll have to see what I can dig up, but I suspect you're way more familiar with OS X than I am.

[Comment created by Pidgeot: 2014-08-31]
I must admit I have very little knowledge on OS X issues like these.

Is this the pre-built executable? Does it work if you build your own? The pre-built one is built on 10.8; that might have something to do with it (...somehow)

Does it generate the stdout.txt and stderr.txt files despite the crash (and do they contain anything)?

[Comment created by fricy: 2014-08-31]
Yes, that is the prebuilt app. the error logs are suposed to be in /Resource, right? because they aren't there... I've experimented with building, but not managed a create a proper one yet. Possibly python should be buillt with deployment target 10.6?

[Comment created by Pidgeot: 2014-08-31]
The error logs will go out next to the app, because they're the primary way the user can report any problems. Hiding them away is not acceptable IMO.

Slight change to new tilesets feature - use /data/art/ instead of LNP/Tilesets

This can cut down the length of the list significantly, and should also save a fair bit of disk space in the pack. We'll also be able to dump text tilesets in the baseline folder to make them available everywhere.

I'll handle this as part of the graphics format changes in my fork - just posting to keep people up to date.

[Issue created by PeridexisErrant: 2014-11-18]
[Last updated on bitbucket: 2015-01-06]

[Comment created by PeridexisErrant: 2014-11-19]
Patch included in pull 10, which has the other graphics format changes I mentioned.

[Comment created by Pidgeot: 2014-11-18]
This doesn't make sense to me. The entire point of Baselines is to have a copy of vanilla DF, if you start adding files, then it isn't.

It also takes away the ability to centrally manage available tilesets. This means that each tileset will only be available by default in that single DF version the original download was configured for. That may not be a problem for most pack users, but as soon as they get a second DF folder, or want to do more things themselves, those tilesets are suddenly not present in that second version - for no good reason.

Finally, I don't see how it's going to save any disk space. Yes, you're avoiding the need to copy the tiles to the art folder, but you still end up storing two copies - one in data/art, one in baselines. Everytime you update DF, you'll need another copy of them in baselines.

[Comment created by Pidgeot: 2014-11-18]
That seems far more reasonable, yes.

[Comment created by PeridexisErrant: 2014-11-18]
The baselines have two purposes: graphics are added to a copy to rebuild, and mods are merged with the raws. I don't think adding tilesets compromises this, but you have a good point about cross-version availability.

How's this for an idea: Use LNP/Tilesets to hold those that need to be added to every pack, then copy them to /data/art during graphics installation. Otherwise you run into the disk space issue, which is that the tilesets folder must contain every tileset used by any graphics pack - as well as them being in at least one pack, meaning ~2x needed space and a very long and confusing list. This way, we'd have a tripartite single source: graphics tiles stored in the graphics pack, vanilla tiles included by the rebuild process, and extra tiles (eg special TwbT fonts) added to all packs when installed from the Tilesets folder. Tilesets are then installed through the customisation menu, as before.

It's a simple patch, so I'll have that up soon.

[Comment created by PeridexisErrant: 2015-01-06]
Closed by changes in pull #10

Show a pop-up on first run to display useful information about the pack

It would be helpful to show information (such as changelog info, tips, etc) when a user runs PyLNP in a new pack revision for the first time. Perhaps it could simply output an external file in LNP/about. Adding a checkbox to confirm that the text has been read might be helpful as well.

For instance, the Dwarf Therapist project has recently changed hands, and Linux appimages are being created now. It would be helpful to inform Linux players of this change so they can make the necessary changes to their systems so the appimage files can read memory properly without causing too much confusion to the players.

[Issue created by carter_scottm: 2017-12-31]
[Last updated on bitbucket: 2018-01-06]

[Comment created by Pidgeot: 2018-01-06]
I suppose we could store the timestamp of some specific text file, and if the file on disk is newer than the stored timestamp, we show it to the user.

I would probably prefer to make the checkbox "never show again", and then instead provide a menu item to display it on demand, in case the user doesn't want to see these pop-ups automatically. Thoughts?

Mark invalid graphics packs in graphics list

If you attempt to install an invalid graphics pack (ie one without the required files), the PyLNP throws an error. Instead, it could:

• Hide the invalid pack from the list.

• Clearly distinguish that the pack is invalid, eg by strikethrough and an explanatory tooltip. (Red text means 'installed' elsewhere, so not that)

Either way the action-time check should remain to catch race conditions, but this would help avoid an occasional problem.

[Issue created by PeridexisErrant: 2014-11-03]
[Last updated on bitbucket: 2014-11-04]

[Comment created by Pidgeot: 2014-11-04]
Basic graphics pack validation during loading of available packs (fixes #37)

→ <<cset 482381bf244b>>

[Comment created by Pidgeot: 2014-11-04]
There's now a basic check on each detected graphics pack which checks for files and directories known to be needed. If a pack doesn't pass that check, it will be excluded from the list.

This, of course, cannot guarantee that installation won't fail for other reasons, but it should at least prevent the most obvious failures.

[Comment created by PeridexisErrant: 2014-11-03]
Before (when the PyLNP is run), so attempting an invalid install is either impossible or clearly inadvisable.

Which we prefer is basically a matter of whether to place responsibility with the maintainer or the user, and I favour the former in both cases.

[Comment created by Pidgeot: 2014-11-03]
You mean it should be marked before attempting an install, or only after a failed install?

Wishlist: Add a linter for raws (in <df>, graphics, mods, baselines, and more)

Implementing a linter for raws would be a really nice feature.

It could be used to check graphics packs, mods, merged mods, distributed raws, and much more!

It's also a good fit for starting with a basic thing with only partial coverage, and expanding the issues it checks as we go. A nice starting point would be to convert Lethosor's raw-lint.lua to Python, and hooking it into our existing modules.

[Issue created by PeridexisErrant: 2015-03-10]
[Last updated on bitbucket: 2015-03-12]

[Comment created by Pidgeot: 2015-03-12]
Basic raw checker based on Lethosor's raw-lint.lua (fixes #73)

→ <<cset 16c2d220e39d>>

Failed to read JSON from PyLNP.user

#!python

Note: Failed to read JSON from PyLNP.user, ignoring data - error details follow
Traceback (most recent call last):
  File "/home/michael/Desktop/df/python_lnp/build/lnp/out00-PYZ.pyz/core.json_config", line 26, in __init__
IOError: [Errno 2] No such file or directory: u'PyLNP.user'

Issue is noted here: Lazy-Newb-Pack/Lazy-Newb-Pack-Linux#6

Found running on Fedora 20.

[Issue created by beaubouchard: 2014-11-12]

[Comment created by beaubouchard: 2014-11-12]
explained Lazy-Newb-Pack/Lazy-Newb-Pack-Linux#6 (comment)

"The path in the stack trace simply comes from the directory the build took place in; it has nothing to do with the file being accessed.

This particular file stores user settings; if this is your first time running the launcher, the message is expected, since you don't have any such settings. (In the event that you did have settings, this message would let you know they couldn't be read, and therefore they weren't applied.)

The message is completely harmless and the program will work just fine without the file. I'll likely change the error message in the near future to give a better message in the case where the file doesn't exist, though."-- Pidgeot

Thanks again.

New DFFD update method throws errors when failing

ERROR: Error downloading http://dffd.bay12games.com/file_data/.json: Not Found
ERROR: Error checking for updates
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Python34\lib\threading.py", line 920, in _bootstrap_inner
    self.run()
  File "C:\Python34\lib\threading.py", line 868, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\~\Documents\GitHub\python-lnp\core\update.py", line 39, in perform_update_check
    if lnp.updater.update_needed():
  File "C:\Users\~\Documents\GitHub\python-lnp\core\update.py", line 143, in update_needed
    self.get_version() != lnp.config.get_string('updates/packVersion'))
  File "C:\Users\~\Documents\GitHub\python-lnp\core\update.py", line 247, in get_version
    self.json = JSONConfiguration.from_text(self.text)
  File "C:\Users\~\Documents\GitHub\python-lnp\core\json_config.py", line 46, in from_text
    return JSONConfiguration(None, json.loads(text))
  File "C:\Python34\lib\json\__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'NoneType'

This is probably because I'm getting the configuration wrong, but there shouldn't be an exception there. As well as checking the None issue, could you provide example valid configs for the Update object in the readme? Mine is below, and I've tried the number as a string and an int without any difference.

  "updates": {
    "updateMethod": "dffd",
    "dffdId": "7622"
  }

[Issue created by PeridexisErrant: 2015-05-03]

[Comment created by Pidgeot: 2015-05-03]
Add error message during updating if pack version is not set (fixes #88)
Add update configuration examples to readme

→ <<cset 8b3d54ff8193>>

[Comment created by Pidgeot: 2015-05-03]
You need to set packVersion - that's what's failing.

I'll add an error message when that one is missing.

Importer not working.

Repost from http://www.bay12forums.com/smf/index.php?topic=126076.msg7218844#msg7218844

The importer isn't working for me, or another user.

I get this message after selecting the old DF folder: WARNING: Can only import content from single basedir.

If I don't select the main DF folder, I get: Import failed. Does not seem to be a DF install directory.

Moving them to the root drive or same folder didn't change anything.

I'm not a coder so I don't understand anything; can't help fix.

Windows 7 x64

[Issue created by DAOWAce: 2016-10-20]
[Last updated on bitbucket: 2017-11-27]

[Comment created by rjpercival: 2017-08-14]
It appears that this section of PyLNP.json is the cause:

"to_import": [
    [
      "text_prepend",
      "<df>/gamelog.txt"
    ],
    [
      "text_prepend",
      "<df>/ss_fix.log"
    ],
    [
      "text_prepend",
      "<df>/dfhack.history"
    ],
    [
      "copy_add",
      "<df>/data/save"
    ],
    [
      "copy_add",
      "<df>/soundsense",
      "LNP/Utilities/Soundsense/packs"
    ],
    [
      "copy_add",
      "User Generated Content"
    ],
    [
      "copy_add",
      "LNP/Utilities/Soundsense/packs"
    ]
  ]

I found that import succeeded if I deleted the last 3 elements of that array.

[Comment created by rjpercival: 2017-08-13]
I too am having this problem. Relevant info:

• Import from: "C:\DF\Dwarf Fortress 0.43.03"
• Import to: "C:\DF\Dwarf Fortress 0.43.05"
• PyLNP: "C:\DF\Starter Pack Launcher (PyLNP).exe" (v0.12c)

I tried moving "C:\DF" to "C:\Games\DF" incase it was related to the depth of the directory tree, but that had no effect sadly.

[Comment created by rjpercival: 2017-08-14]
The logging provides a clue:

INFO: Importing from  to
WARNING: Can only import content from single basedir

[Comment created by DAOWAce: 2016-10-24]
M:\DF\Dwarf Fortress 0.42.06
M:\PeridexisErrant's Starter Pack 0.43.03-r07\Dwarf Fortress 0.43.03

PyLNP in the base folders for each.

M:\Dwarf Fortress 0.42.06
M:\Dwarf Fortress 0.43.03

Single PyLNP in the root directory.

Drive letter doesn't seem to matter, was previously on C: and in a much deeper folder structure on my desktop (ex; C:\Users\Owner\Desktop\crap\DF...)

[Comment created by jecowa: 2016-10-24]
That's a good theory. I tried moving both Newb Packs from the /Applications/ folder (where they were giving me problems) to the ~/hg/ folder and I got a different error message this time:

#!alert error

Import failed.

Nothing is configured for import in PyLNP.json

See the log for more details.

Then I installed the default PyLNP.json from version 0.12a, and it fixed it.

Tl;dr - The import feature doesn't work in the "Applications" folder, but it works in the "~" folder (my user directory folder).

My user directory folder is located at "/Users/jecowa/" in case that matters.

[Comment created by jecowa: 2016-10-20]
It's not working for me either on Snow Leopard. I'm not sure what the deal is. It was working fine before when running from source in Yosemite. Maybe it's only failing under certain circumstances. The "DF" folder is definitely the right one, though.

[Comment created by Pidgeot: 2016-10-23]
Could either of you please post the following information? It might help track it down.

• Full path you're trying to import from
• Full path you're trying to import to
• Location of PyLNP

Feel free to anonymize path components, as long as the number of path segments is the same (so /a/b/c/d is acceptable as a substitute for /home/myname/games/df, but /a/b/c would not be).

@PeridexisErrant I suspect this is os.path.dirname that's not behaving like you're hoping it does, so if that tells you what's wrong, then feel free to fix, of course - otherwise I'll try to take a look when I can (though it may take a little while)

[Comment created by Pidgeot: 2017-08-29]
@RJPercival Thank you for helping track down this bug. I will attempt to take a look at this soon; haven't really had much time to work on PyLNP lately...

[Comment created by Pidgeot: 2017-09-17]
Fix path manipulation for LNP-specific paths in importer (fixes #134)

→ <<cset 440758e0f55e>>

[Comment created by rjpercival: 2017-09-17]
Thanks for fixing this :)

[Comment created by Pidgeot: 2017-11-27]
Removing milestone: 0.12d (automated comment)

Updating mechanism

It would be nice to have some form of in-program autoupdating, with hooks to allow pack creators to have their packs downloaded.

[Issue created by Pidgeot: 2014-07-18]
[Last updated on bitbucket: 2014-08-15]

[Comment created by Pidgeot: 2014-08-15]
Update checking (fixes #2)

→ <<cset 774788339710>>

.bat-based utilities aren't detected by the launcher

Utilities that are launched by a .bat file (e.g. World Viewer) won't get listed on the utilities page because it only looks for .exe and .jar files on Windows.

[Issue created by PowellNGL: 2014-07-30]
[Last updated on bitbucket: 2014-08-15]

[Comment created by Pidgeot: 2014-07-31]
I left out .bat files by default, because it seemed more likely that they would provide false positives - some utilities might generate .bat files during runtime, and you might not know in advance what those files will be called, so they can't be filtered out using exclude.txt.

You can create a file LNP/Utilities/include.txt to force certain file names to be included, if they exist. The format is the same as exclude.txt in the same folder (from the current starter pack) - everything between brackets is treated as a filename, everything else is a comment.

That said, if people want .bat to be included by default, that's fine with me.

@jonathan_bennett: You can fork the repository and submit a pull request, or you can make a patch file, and then you should be able to attach it under the More button in the top-right corner.

[Comment created by jonathan_bennett: 2014-07-31]
Pidgeot, I just put together a quick patch to fix this. But this being my first time on bitbucket, I've got no idea how to submit the patch...

[Comment created by Pidgeot: 2014-08-15]
Also show .bat files as utilities on Windows (fixes #6)

→ <>

Error on launch during update check

From /u/ciphertext008 on Reddit:

#!python

Traceback (most recent call last):
  File "<string>", line 8, in <module>
  File "C:\Users\<user_name_stripped>\Documents\LazyNewbPack\python-lnp\build\lnp\out00-PYZ.pyz\lnp", line 88, in __init__
  File "C:\Users\<user_name_stripped>\Documents\LazyNewbPack\python-lnp\build\lnp\out00-PYZ.pyz\lnp", line 746, in check_update
TypeError: coercing to Unicode: need string or buffer, error found

[Issue created by Pidgeot: 2014-09-19]

[Comment created by Pidgeot: 2014-09-19]
Unable to reproduce, but I think I fixed it based on the stack trace.

Check if utility is already running before launching?

This may be to much of a PITA to do cross-platform, but I'll pass on the request anyway - I've been asked a couple of times.

The windows launcher was fairly recently updated to avoid launching a program that was already running - if you try, it pops up a confirm/cancel prompt that defaults to 'no' after twenty seconds. Implementing this in the PyLNP would be appreciated.

[Issue created by PeridexisErrant: 2014-09-18]
[Last updated on bitbucket: 2014-09-19]

[Comment created by Pidgeot: 2014-09-19]
If a program is running, a window like this will now pop up:

It currently shows the full path to make sure the user knows what's being run; I may still choose to cut off most of that (but I haven't decided how much).

The window will automatically close after 20 seconds; only then will it try to launch any further applications.

[Comment created by Pidgeot: 2014-09-19]
Prompt to re-run already running programs (fixes #23)

→ <<cset 811af0685e3d>>

[Comment created by Pidgeot: 2014-09-19]
I've managed to implement the simple solution (cset db95f9b0ca75), and am able to handle point 2 by parsing the output of the ps command on those platforms. That's good enough for me.

Currently it just flat out refuses to run anything it knows is running; there's no mechanism for forcing a new instance. I'll be looking at that later today.

[Comment created by PeridexisErrant: 2014-09-18]
1 is definitely fine, and 2 is still better for utilities and Windows than the current situation.

[Comment created by Pidgeot: 2014-09-18]
This one is slightly problematic because AFAICT Python doesn't have a built-in way of detecting all running processes.

A simple solution would be to simply check if the launcher itself launched the program previously, and if so, whether that program is still running - because that's something I can do. However, that does mean two things:

1. If the launcher is closed and re-opened, it will lose this information (so any utility started manually or in a previous session won't be detected)
2. On Linux and OS X, I don't think it will work for DF itself when DFHack is in use, since those use a secondary application to start the program, and I think they appear to have terminated as soon as the new process is started.

1 is probably acceptable, but 2 probably isn't.

I'll have to do some testing to see if that one really is a problem, but if it is, I'll need to write platform-specific code (I have a decent idea about what Linux and OS X would need; I'm less sure about Windows).

Graphics install broken for some users

http://www.bay12forums.com/smf/index.php?topic=126076.msg6069660#msg6069660

It seems that in 0.9.2a, and .3, the graphics install logic works up to removing the raws and then fails to copy the raws in baselines/temp/. This might be fixed by replacing dir_util.copy_tree with shutil.copytree, but since I can't replicate the bug I'm hesitant to try.

[Issue created by PeridexisErrant: 2015-03-01]
[Last updated on bitbucket: 2015-03-02]

[Comment created by PeridexisErrant: 2015-03-01]
Looks like it does fix the main bug - the other problem is that Windows is weird about what operations it allows when a dir is open in file explorer (you can delete but not copy???).

I'll check that and fix a few other potential issues I spotted in unusual cases, then make a pull request.

[Comment created by PeridexisErrant: 2015-03-02]
Fixed, for the bug responsible this time, by pull #32

[Comment created by Pidgeot: 2015-03-01]
I'll give you a little while to gather up some results.

Also, even if it isn't enough, it seems like switching to shutil might be a good idea anyway, considering the issues we've seen from dir_util.

[Comment created by PeridexisErrant: 2015-03-01]
This might be fixed by recent changes in my repo. I've asked for testers, and will make a pull or revise once I hear back.

Improve merging - experiments with total conversions

@PeridexisErrant
I've been running some test to see how the mod merger works on merging graphics into total conversions. I used 3 mods for this test, Accelerated Modest Mash, Button's Modest Mod and Dark Ages. All these three has an ASCII branch and at least one graphic branch too, so they seemed ideal.
Methodology:
First I switched to an ascii set, the reason being that the mod merger complained with the color orange if I wanted to just merge a mod on top of a graphic modified raws like Phoebus. When the mod merging was done I switched the graphics and run a diff to compare the result with my control raws.
Improvement: Imho the merger shouldn't try ever to merge modded raws into graphics modified raws, but it should always start merging/assembling from scratch (baseline raws), and when content merging (mods) is done it should do another pass of display merging (graphics).

1st run: Accelerated-Modest-Mash + Phoebus
Results:
'raw/objects/creature_small_ocean.txt' are different
'raw/objects/creature_standard.txt' are different
'raw/objects/creature_subterranean.txt' are different
'raw/objects/inorganic_stone_gem.txt' are different
'raw/objects/inorganic_stone_layer.txt' are different
'raw/objects/inorganic_stone_mineral.txt' are different
'raw/objects/inorganic_stone_soil.txt' are different
'raw/objects/item_tool.txt' are different
'raw/objects/plant_grasses.txt' are different
'raw/objects/plant_new_trees.txt' are different
'raw/objects/plant_standard.txt' are different
'raw/objects/reaction_other.txt' are different

Creature files are mostly ok, 3 errors in 3 files.
Inorganics are plain wrong. The mod removes quite a few of them, and it looks like graphics merging brings them back.
Item tool seems like ASCII, beats me why it hasn't been processed.
Plant grasses: removed grasses are added back by merging - just like with inorganics.
Plant_new trees: Tile tags are not merged correctly, may be improved by changing some lines in Phoebus. Curiously standardized tree materials seem to be merged ok in this file...
plant_standard: Standardized tree materials have lot"s of errors along with some graphics tags. Needs attention.
reaction_other: mod custom reactions are gone.
The Mod is difficult to process, uses a lot of tricks to standardize materials in creatures and also in plants. Changed tissues are processed correctly about half of the time, but as mentioned removed content is added back by the graphics merge.

2nd run: Buttons new Modest Mod + Phoebus
'raw/objects/creature_subterranean.txt' are different
'raw/objects/plant_standard.txt' are different
'raw/objects/reaction_other.txt' are different

Very good result, GCS is missing the child and plump helmet tiles are a bit messed, but nothing serious. Impressive. It might have to do with Button's raws being cleaned/closer to vanilla compared to Acc-Modest.

3rd run: Dark Ages + Ironhand
I've spent some time figuring out what's up with this one, turns out the Ironhand version is not 100% Ironhand merge. Apart from this it looks like most of the files are OK, except for inorganic_stone_mineral.txt which is pretty badly messed up, I think even broken. I haven't tried generating a world with it to see what'd happen, but doesn't look good.

Conclusion: Merging works quite well when processing raws with matching structure, but fails when lines/blocks or text are missing. (like removed materials in mods)

Improvement Ideas:
Some of these problem originate from doing a full merge of the graphics into already modded raws. One solution is to only try to merge the difference between baselines and graphics, which would ideally result in merging only the graphic part of the raw/objects, and leave the content part of the raws out.
A different take on this idea is to limit the graphic merge to the lines or even better, to [CODE_BLOCKS: ] with tags that we already know are to do with changing the display. These tags are the following:

creature raws:
[CREATURE_TILE:224][COLOR:3:0:1]

inorganics raws:
[DISPLAY_COLOR:7:12:0]
[TILE:153]
[ITEM_SYMBOL:'*']

tools:
[TILE:153]

grasses:
[GRASS_TILES:231:159:231:159]
[GRASS_COLORS:2:0:1:2:0:1:2:0:1:6:0:0]

plant raws:
[GROWTH_PRINT:0:6:2:0:0:ALL:1]
[PICKED_TILE:58][PICKED_COLOR:2:0:0]
[TREE_TILE:5]
[SHRUB_TILE:58]
[DEAD_SHRUB_TILE:58]
[SHRUB_COLOR:5:0:0]
[DEAD_SHRUB_COLOR:0:0:1]

[Issue created by fricy: 2015-02-10]
[Last updated on bitbucket: 2015-03-13]

[Comment created by PeridexisErrant: 2015-03-13]
Some mods logging is now included (see f4daa4f). Further discussion of that can move to issue #70 - and specific suggestions for or contributions to mods logging are welcome.

Improvement of graphics merging with mods can be discussed in issue #74. Most of my ideas go "x, and then magic".

Manifest issues go in issue #71.

Hopefully that covers everything - if not, open a new issue for the specific suggestion or problem.

[Comment created by PeridexisErrant: 2015-03-13]
Graphics with mods are now optional, fixing point #4 above.

Next I'll work on logging, then close this issue and open one to upgrade the graphics install to a more intelligent last-on magic solution.

[Comment created by BoomButton: 2015-03-09]
I third fricy's suggestion for graphics merging; actually, I logged in tonight to suggest the very same thing!

Just to check in, I've been working on an Accelerated version of the Modest Mod. The graphics pack merging really doesn't do well with mods that remove big chunks of the vanilla raws. For example, the accelerated mod removes all the plant growths that dwarves can't use - and the absence of sometimes 20 lines at a time, in a file in which a lot of lines are essentially identical across objects (e.g. tree raws), screws the diff-based merging up pretty badly.

[Comment created by PeridexisErrant: 2015-03-02]
Excellent. I'll be glad to avoid working on logging for as long as it takes you ;p

The manifest could include metadata such as author, url, mod version, DF version, mod name, simplification state, whether it includes graphics other dfhack stuff, whether it's a "major" or a "mjnor" mod, and so on. Some would be purely cosmetic, but there's a lot that we can do with info that's either not available otherwise or impractical to get.

Yep, something like Fricy's idea. Basically a with in-line merge for lines with certain tags. It would probably require some context though, which almost makes it a special case of the mythical raw parser...

[Comment created by Pidgeot: 2015-03-02]
@PeridexisErrant

1.The mods merge should output a log of every change with status != 0. This is conceptually simple but a pain to implement, since we'll want the problem lines, line numbers, and filenames.

I have some plans about adding better logging in general, so at the very least, it should become easier to do that once I have time to implement that.

2.Implement a manifest.json for each mod, including all metadata not apparent from simple inspection. The system must handle manifest-less mods with sensible defaults, and should be extensible to utilities and graphics.

It would be helpful to know what you're expecting such a manifest to contain, because it's not at all certain I'd have the same idea.

3.Adding graphics to mods could be changed from a first-on simple overwrite, to last-on regexp magic. I can't do this myself, but it seems like it might solve a lot of problems (and create many, too).

Just to be clear, are you thinking of something along the lines of fricy's last comment, or something else? (Raw parsing is, of course, still the dream...)

[Comment created by PeridexisErrant: 2015-02-22]
This is disheartening news in general - it implies that the merge of overlapping sections is less robust than I thought, and that was already enough for an orange merge warning.

The general advice to keep mods as close to vanilla as possible is sound though, and as noted there's not much that wee can do short of full raw parsing - which is well beyond my ability to set up.

[Comment created by fricy: 2015-02-21]
I did some experiments with @BoomButton 's Mod to figure out the merge problems with the GCS child and the Plump Helmet. What follows is some advice to work around the limitations of the process.

GCS: I'm fairly certain that the modification is lost because it's directly above the line with the CREATURE_TILE. When merging the graphics Pylnp seems to discard this line. Moving it after creature tile seems to work, but I think it's even better keep your modifications (in a block) close to each other whenever possible. So in this case as you add a population number too maybe this is best starting from line 1050:

[LARGE_ROAMING][FREQUENCY:20][DIFFICULTY:3]
[POPULATION_NUMBER:30:40]		Modest Mod
[CHILD:2][GENERAL_CHILD_NAME:spiderling:spiderlings]		Modest Mod
[CARNIVORE][NATURAL]

So my 2c: If possible avoid editing lines above or below tags that are (usually) changed by graphics, and if possible keep your edits close to each other. This applies to other animals (eg: turtles) as well, as when I checked the Spacefox merge these were affected too.

Re: Plump helmets: the problem here was that you changed the baseline values to something else, and when the graphics merge phase was done it couldn't find the appropriate lines and the merge got confused. It looks like you reverted the tile changes since, so now it works ok. I don't have any other advice besides not to do any graphic editing in the mod, and keep everything as close to ascii/baseline as possible.

RE: Growth_print and other tags:
Looks like it's going to have to be raw parsing then if supporting total conversions is a goal.
Although perhaps as an interim step some regex based magic could be used (only for graphics) to merge only the lines with the tags above, and that means that the graphic pack could be further reduced in size to something like this:

creature_subterranean  
[OBJECT:CREATURE]
[CREATURE:CROCODILE_CAVE]  
    [CREATURE_TILE:'C'][COLOR:7:0:0]

[CREATURE:TOAD_GIANT]  
    [CREATURE_TILE:'T'][COLOR:2:0:0]  

etc.

Then you locate the line with the appropriate creature, and look for the next line containing the tag. And so on...
It still wouldn't help with the growth_print tag, but at least would stop deleted inorganics from being merged back into the raws.

[Comment created by PeridexisErrant: 2015-02-12]
Oh, joy. Still, the current system should handle that as well as it does anything else. So long as it's not re-modified by a mod, the graphics will be fine, and it's not clear what "should" be the case if it is modified.

[Comment created by BoomButton: 2015-02-12]
Hey.

I just wanted to point out the big snag around plant raws, which is the GROWTH_PRINT tag.

The GROWTH_PRINT tag combines graphics information with information about in what season and order the graphics need to appear. To illustrate, here's the growth print for bananas (the fruit, not the whole plant):

[GROWTH_PRINT:'%':'%':6:0:1:120000:200000:3]

The first five fields - '%':'%':6:0:1 - are the ones that graphics packs would be concerned with. The remaining three (sometimes two, the second field is optional) fields are about when the growth appears in the year, and how it relates to other growths on the same tile.

So, if I wanted to make bananas available for twice as long each year, I would do that by changing that tag to [GROWTH_PRINT:'%':'%':6:0:1:12000:38000:3]. It's not a graphics change, but a dumb graphics applicator will treat it like one.

This is a pretty big deal for any mod that touches plants, as every growth has at least one GROWTH_PRINT tag (and plenty have more than one!). But, plants aren't extremely popular things to mod, so you may consider plants as a whole an edge case.

As for converting graphics on new/modded-in plants and growths, these usually follow a pretty strict pattern, to the point where I suspect if you did a statistical comparison between default raws and graphics'd raws, you'd get a clear pattern of which tile converted to which tile 90% of the time. Which would give you a pretty good guess of what tiles to convert to what. But, that's a stretch goal for sure.

[Comment created by BoomButton: 2015-02-13]
If the first 5 fields are changed I agree that the ideal outcome isn't clear; but I think that if the first 5 fields are pristine, the applicator "should" ideally be able to swap out the graphics-related fields for the values of other graphics packs.

The next iteration of Plant Fixes is going to change those last 2-3 values for the grand majority of GROWTH_PRINT tags in the raws. It won't be ready for another few months, but when it is, it's gonna be a doozy.

[Comment created by PeridexisErrant: 2015-02-11]
What's meant to happen: Copy baseline, copy graphics over, merge mods into this base in order. The load order is graphics - mod1 - mod2 - etc. It sounds like you merged raws mod1-mod2- graphics. This is important because later changes overwrite earlier ones!

I think what happened with, for example, the messed up inorganics was that merging graphics put the non-vanilla lines back, where the graphics lines should have been removed with the vanilla lines by the last-on mod. (Overwriting like this is what gives orange status, since we can't test for a valid result yet - just a heuristic based on process).

The really dumb solution would be a pain to implement - currently it's appending slices of lists, which could be any number of lines. This would also make a detailed log a real pain to work with.

A manifest.json will indeed be required eventually, though it won't need a list of filenames (see below). For the rest though, I think it would be quite valuable - and worth extending to graphics and utilities, as well as mods. Details should wait for a while though - I've contacted Janus, and DFFD will be upgraded to give each file a json dump of all the associated metadata. Since we probably want that as a wholly included subset, I'm planning to work on the manifest after that's done.

As to listing filenames: that's handled with a fairly minimal assumption in the format. If you want to have files truly deleted, they must be replaced by empty files - and this is implemented by the simplification logic. If the first step discards ten or more files (ie it's a full install), any missing vanilla files will have blank placeholders created before vanilla files are removed. If mod authors distribute in more complex formats, it becomes the responsibility of the person adding the mod to ensure that it's formatted correctly.

[Comment created by fricy: 2015-02-11]

Un-removing inorganics is why the order is important: later changes override earlier ones, so graphics should go first. I suspect many issues can be traced back to this. Could you give this another go using the default load order? I'm hoping that will work better...

I don't understand what you mean by this. I only tried loading one mod at a time with the baselines (ascii) and then apply a graphic pack on top. There's no load order or whatever.
EDIT: You mean I should try applying the mod on top Phoebus raws? I tried, surprisingly there's no difference between the two results. I'm a bit confused.

This is why you got an orange result - the first mod was further changing lines which were different from vanilla.

Yes, I figured it out. That's why I'm proposing that behind the scenes the mod merger should target the baseline raws, and not the graphics modified raws in df/raw. No more orange. After pressing the install mod button do the merge and examine the state of the graphics tab and apply the selection.
EDIT: I'm not sure what happens behind the scenes now. :)

A system that only changes certain tags would seem to require a whole new raw-parsing system, which I can't build - though it would be nice.

Really dumb solution (without having to parse things): do the (graphic merging) process as before, but examine each merged line. If the line in question contains any of the tags in my earlier post use the merged line, otherwise don't change it.

Other ideas:
1, Having a log file of the merge would be really-really-very-very helpful for troubleshooting and bugfixing. We don't need to know about succesfull lines, but the merge conflicts, problematic lines are needed preferably with filenames and line numbers.
2, I think we are going to have to require a manifest with the mods. It should contain mod name, mod filenames (according to Deon some mods remove vanilla files, which the baselines would put back, http://goo.gl/nZF5Iw), mod version, mod compatibility version (to know when can we upgrade), and a description that could be used as a tooltip/popup/help/whatever. JSON format?

[Comment created by PeridexisErrant: 2015-02-10]
Hmm. Given the scale of total conversion changes, that's not too bad!

The current graphics merging logic is quite simple: files in the pack to install replace those in baselines/temp, and mods are then merged as usual. This assumes that graphics raws aren't too different from mod raws, which seems reasonable, and that the first mod can be merged simply by replacing files, which is true as a special case of the merge logic - we could call the full merge functions with the same result.

This is why you got an orange result - the first mod was further changing lines which were different from vanilla. A system that only changes certain tags would seem to require a whole new raw-parsing system, which I can't build - though it would be nice.

Un-removing inorganics is why the order is important: later changes override earlier ones, so graphics should go first. I suspect many issues can be traced back to this. Could you give this another go using the default load order? I'm hoping that will work better...

I discussed this briefly with Button recently, and noted that there are a few changes in Modest Phoebus not in either component - namely the tiles for Plump Helmets. How to deal with this kind of edge case is something I'm still thinking about.

[Comment created by PeridexisErrant: 2015-03-02]
List of concrete ideas that should help:

1. The mods merge should output a log of every change with status != 0. This is conceptually simple but a pain to implement, since we'll want the problem lines, line numbers, and filenames.

2. Implement a manifest.json for each mod, including all metadata not apparent from simple inspection. The system must handle manifest-less mods with sensible defaults, and should be extensible to utilities and graphics.

3. Adding graphics to mods could be changed from a first-on simple overwrite, to last-on regexp magic. I can't do this myself, but it seems like it might solve a lot of problems (and create many, too).

4. Including graphics in the mods to merge should be optional in general, for maximum compatibility. This would with the current setup be done entirely in tkgui/mods.py

I'm not promising to work on any of these, but aside from fixing bugs as they come up these ideas seem like the logical next step.

Improve handling of TwbT graphics files

Unfortunately the installation of TwbT graphics files in 0.13a (from 18b83e) does not work.

• the function points to nonexistent paths, and so doesn't copy anything at all
• string replacement on paths is unsafe; we should use os.path.relpath instead

[Issue created by Former user: 2018-09-01]
[Last updated on bitbucket: 2018-09-17]

[Comment created by Pidgeot: 2018-09-01]
Any chance you could paste the error log for reference?

[Comment created by PeridexisErrant: 2018-09-01]
There's no error log, it just doesn't copy the files under data/art/ and data/init/.

for folder in ['graphics', 'objects']:
    # twbt_folder = paths.get('graphics', pack, 'raw', 'twbt_'+folder)
    twbt_folder = paths.get(folder, '..', 'twbt_'+folder)  # I think this is the subdirs of 'raw', right?
    target_folder = paths.get(folder)
    for path, _, files in os.walk(twbt_folder):
        for f in files:
            twbt_f = os.path.join(path, f)
            # target_f = os.path.join(target_folder, os.path.relpath(twbt_f, twbt_folder))
            target_f = twbt_f.replace(twbt_folder, target_folder)
            shutil.copyfile(twbt_f, target_f)

[Comment created by Pidgeot: 2018-09-01]
Ah, yeah, it might be looking in the wrong folder. Oops.

@PeridexisErrant: If you want to make that change you're suggesting in the commented out code, feel free; I can push out a new build easily if this gets fixed. If not, I'll try to take a closer look in a few days (when I have time to investigate a bit more).

[Comment created by PeridexisErrant: 2018-09-01]
I've been meaning to take a close look at this since late July so you might still get there first; the comments are a start but it'll need some investigation and more importantly a quick test-drive!

[Comment created by Pidgeot: 2018-09-17]
So I finally got a chance to look into it, and yeah, I goofed horribly. My bad.

I've double-checked everything this time, and it definitely works now. I'll push the change and start the builds straight away (0.13b).

[Comment created by Pidgeot: 2018-09-17]
Fix TWBT file replacement handling (fixes #159)
Bump version number to 0.13b

→ <>

Text baseline in tabs on macOS is too high, affecting readability.

While the text is fine in buttons, text fields, tables, and other items, the text in the tab headers near the top of the launcher appears crammed, and the baseline is substantially higher than in buttons.

In the screenshot attached, it almost looks like the text is centered correctly relative to the shaded box, but the button outlines are drawn too low.

PyLNP 0.13, macOS 10.13, DF v0.44.05

[Issue created by alexchandel: 2018-02-20]

Links from external file

Allow pack distributors to customize the links menu without having to edit source, etc.

This would be a simple text file containing URLs and a short description, and some way of adding a separator.

[Issue created by Pidgeot: 2014-07-18]
[Last updated on bitbucket: 2014-08-15]

[Comment created by PeridexisErrant: 2014-07-23]
It would also be good to do the same thing for the folders menu, for much the same reasons.

In the windows launcher these are both done through a file LNPWin.txt (see here). For this pack, there's likely to be feature creep in terms of the volume of things configured (eg links / folders / update info), so it might be good to jump directly to something structured, like the xml config for Soundsense.

[Comment created by Pidgeot: 2014-08-15]
Move folders and links to external JSON file (fixes #3)

→ <<cset 85ee176fb6d3>>

Automatically install raw/twbt_* over main raws if TwbT is enabled

Several graphics packs have adopted a split structure, to better support new TwbT features without making it unusable for players who are not using TwbT.

On installation of a graphics pack when TwbT is enabled (or on enabling TwbT), we should copy these files into place over the base raws, and when TwbT is disabled we should reinstall the graphics without them.

As a short-term workaround, I plan to bake these changes in to the graphics provided in my pack as TwbT is strongly recommended for new players, but it would be better to support them properly via PyLNP.

https://github.com/DFgraphics/Spacefox/tree/master/raw
PeridexisErrant/starter-pack#24

[Issue created by PeridexisErrant: 2018-05-21]
[Last updated on bitbucket: 2018-07-08]

[Comment created by Pidgeot: 2018-07-08]
When installing graphics pack, use matching twbt_* folder to override files if TWBT is enabled (fixes #155)

→ <<cset 18b83e23ff5e>>

[Comment created by Pidgeot: 2018-07-08]
I've implemented the part where installing a graphics pack will use the overrides, but I did not implement the part where changing whether or not TWBT is on will reconfigure the graphics pack, since that is going to put quite a big overhead on initial configuration, and I'm not sure it's worth it.

Feel free to re-open the issue if you disagree, then we can look into that later.

DFHack files in /raw/ and the mods merge logic

The mods tab should handle DFHack files reasonably correctly. This will require making three general changes:

• If a file does not exist in vanilla DF, run through the merge logic with an empty string in place of the vanilla file and save to the mod filename.

• Add last special case to mods.do_merge_seq(); if mod==gen, return the genned lines instead of merging (ie do not change duplicate files).

• As well as .txt allow merging of .init, .lua, and .rb files.

I'll do this one by mid next week, hopefully. (It would be good if the 'assign to me' button worked)

[Issue created by PeridexisErrant: 2015-01-23]
[Last updated on bitbucket: 2015-02-05]

[Comment created by Pidgeot: 2015-02-05]
Merged in PeridexisErrant/python-lnp (pull request #21)

Closes #56, #61

→ <<cset 399de4d27a76>>

0.9 x64 version crashes on dfhack start

:D Hi I just wanted to document a crash I got while testing.

I was testing the new LNP configuration on Ubuntu with with the PyLNP 0.9 linux-x64 version and DFHack 0.40.24-r0 when I encountered a crash. When trying to start DFhack through the GUI, it crashes silently and dwarf fortress never starts; no windows appear. I tested it with both the i686 and x64 versions, same error. Dfhack and DF each work independently from the gui.

Here is the error I found in Output log:

#!python



Exception in Tkinter callback

Traceback (most recent call last):

  File "/home/michael/Desktop/df/python_lnp/build/lnp/out00-PYZ.pyz/Tkinter", line 1486, in __call__

  File "/home/michael/Desktop/df/python_lnp/build/lnp/out00-PYZ.pyz/core.launcher", line 46, in run_df

  File "/home/michael/Desktop/df/python_lnp/build/lnp/out00-PYZ.pyz/core.launcher", line 83, in run_program

  File "/home/michael/Desktop/df/python_lnp/build/lnp/out00-PYZ.pyz/core.launcher", line 123, in program_is_running

TypeError: decode() argument 1 must be string, not None

Distro:

• Ubuntu 14.04.1 LTS -- trusty

Let me know if there is anymore information you need, or anything else I can do.

Thank you so much for all your hard work and releasing cross platform to linux!! :D

[Issue created by beaubouchard: 2015-01-24]
[Last updated on bitbucket: 2015-02-05]

[Comment created by Pidgeot: 2015-02-05]
Use a fallback encoding when checking for running processes with ps (fixes #57)

→ <<cset 175ca30253f5>>

[Comment created by Pidgeot: 2015-02-04]
So, the thing it's complaining about it this line:

    return path in s.decode(sys.getfilesystemencoding())

Specifically, it's saying that on your systems, sys.getfilesystemencoding() is returning None, at least when run from the binaries. Apparently this will happen if the system doesn't specify a specific encoding.

I'm adding a fallback to UTF-8 for that particular case, and telling it to ignore errors during decoding (in the rare event of a non UTF-8 *nix/OSX).

[Comment created by Pidgeot: 2015-02-05]
Actually fix #57

→ <<cset 05b0ae1bd09d>>

[Comment created by cptmashek: 2015-01-28]
This also happens in Linux Mint Debian Edition, amd64.

[Comment created by beaubouchard: 2015-01-26]
also happens with DFHack 0.40.24-r1

Clarify update prompt wording and interaction

Several people have reported concern due to the wording in the update prompt - specifically that clicking 'update' will automatically download the new version and may overwrite user data. Changing the text of the prompt from Update now? to Download now? would clarify this.

The relative placement of the check again options and yes/no buttons can also be confusing, as it seems to be confirming the time to check again.

Unlike the other dialogue boxes (eg for graphics install), the update prompt always appears in the top-left of my screen rather than in the centre.

I propose a couple of interface enhancements:

• The update prompt should be centred, like the others

• If an update is available, show a dialogue box with text 'An update is available (version $ver_str)' and two buttons, Download and Not now - avoiding use of the more general verb 'update'.

• If Download is clicked, open the page (nb - could direct download as an extra enhancement later)

• If Not now is clicked, then a new dialogue box opens and prompts the user to choose when next to be reminded. A dropdown menu would be a poor choice here as space is not limited and visibility would be good - I would suggest a set of exclusive checkboxes and either go with the first clicked or have an OK button at the bottom.

[Issue created by PeridexisErrant: 2014-09-23]

[Comment created by Pidgeot: 2014-09-23]
Improve update dialog (fixes #30)

→ <<cset 2f115969cf04>>

[Comment created by PeridexisErrant: 2014-09-23]
Yep, that sounds better than a second dialogue window. The other parts stand though..

[Comment created by Pidgeot: 2014-09-23]
Technically, we don't really need the dropdown anymore; it's a menu item now. Perhaps it would be simpler to just replace it with a note saying how to change it? Something like:

You can control the frequency of update checks from the File > Check for Updates menu.

Request: Command line flag to immediately launch Dwarf Fortress

It'd be a convenience to have a command line option (--launch, for example) that would immediately start up Dwarf Fortress as configured, without opening up the PyLNP GUI

[Issue created by thetoastcaper: 2018-10-12]

Mac - seemingly random graphics pack gets outlined after switching graphics packs

For some reason Curses has an outline after switching to DawnFortress.
example on imgur

[Issue created by jecowa: 2016-08-09]
[Last updated on bitbucket: 2018-01-06]

[Comment created by jecowa: 2016-08-25]
It does this in previous versions too. It seems to be highlighting the graphics pack based on the mouse position when I click the "Okay" button.

[Comment created by jecowa: 2016-08-09]
It doesn't do this if I use the "return" key instead of the mouse to hit the two "Okay" buttons. Also, this problem doesn't exist on either of the tileset selectors.

[Comment created by Pidgeot: 2018-01-06]
As best as I can tell, this is an artifact of how TKinter works on Mac and doesn't seem to be anything that can really be fixed on PyLNP's end.

Add window size option to advanced tab

Drengor:

One thing I'd like to see, on the graphics tab or advanced tab, is the ability to set the x/y. I always use 80/50, and it's the one thing I have to hand edit the inits for every upgrade of LNP.

This is a setting from init.txt;

This is the size and font for windowed mode. Changing to the 800x600 font might make you happier. If set below 256x256 it specifies the grid size instead, with a minimum of 80x25.

[WINDOWEDX:80]
[WINDOWEDY:25]

This should be a text field (as FPS is), but limit entries to integers of at least minimum size, and at most 255.

[Issue created by PeridexisErrant: 2015-02-17]
[Last updated on bitbucket: 2015-04-12]

[Comment created by Pidgeot: 2015-04-12]
This adds options for both windowed and fullscreen resolutions, and it also keeps the GRAPHICS_* options in sync with the normal options (the normal options will be used if they do not match, since this also ensures compatibility with old versions that do not have the GRAPHICS_* options).

I have decided not to cap it at 255 to allow better control for handling fullscreen resolutions.

[Comment created by Pidgeot: 2015-04-12]
Add resolution options to advanced tab (fixes #64)
Automatically synchronizes GRAPHPICS_* resolution options with normal options

→ <>

[Comment created by PeridexisErrant: 2015-03-15]
I couldn't work out what was going wrong with this, but for what it's worth my attempt is below:

diff --git a/tkgui/advanced.py b/tkgui/advanced.py
--- a/tkgui/advanced.py
+++ b/tkgui/advanced.py
@@ -30,6 +30,8 @@
         self.volume_var = StringVar()
         self.fps_var = StringVar()
         self.gps_var = StringVar()
+        self.winX_var = StringVar()
+        self.winY_var = StringVar()
 
     def create_controls(self):
         Grid.columnconfigure(self, 0, weight=1)
@@ -69,17 +71,26 @@
                     'will be less reponsive.').pack(anchor="w")
 
         if lnp.settings.version_has_option('introMovie'):
+            window = controls.create_control_group(self, 'Window')
+            main_grid.add(window)
+            controls.create_option_button(
+                window, 'Windowed', 'Start windowed or fullscreen',
+                'startWindowed').grid(column=0, row=1, sticky="nsew")
+            Label(window, text='Window x dimension').pack(anchor="w")
+            #controls.create_numeric_entry(
+            #    window, self.winX_var, 'winX', 'tooltip')
+            Label(window, text='Window y dimension').pack(anchor="w")
+            #controls.create_numeric_entry(
+            #    window, self.winX_var, 'winY', 'tooltip')
+
+
             startup = controls.create_control_group(self, 'Startup')
             main_grid.add(startup)
             Grid.columnconfigure(startup, 0, weight=1)
-
             controls.create_option_button(
                 startup, 'Intro Movie',
                 'Do you want to see the beautiful ASCII intro movie?',
                 'introMovie').grid(column=0, row=0, sticky="nsew")
-            controls.create_option_button(
-                startup, 'Windowed', 'Start windowed or fullscreen',
-                'startWindowed').grid(column=0, row=1, sticky="nsew")
 
         saverelated = controls.create_control_group(
             self, 'Save-related', True)

[Comment created by Pidgeot: 2015-02-17]

The problem here is that larger values actually have a dramatically different function: they set the target dimensions for each individual tile in pixels, rather than window dimension in tiles

Huh? Setting > 255 just means that's the window size in pixels - and that's particularly useful for e.g. fullscreen, where you'll normally want to hit your monitors native resolution exactly, rather than having to do the division (which you'd need to redo if you switch graphics packs).

Also, I just remembered we'll need to deal with the GRAPHICS*X/Y settings as well - they should probably be kept in sync with the regular ones.

[Comment created by PeridexisErrant: 2015-02-17]
A button to toggle window/fullscreen (windowed/ask/fullscreen; init values are different but only clear from context) would make good sense.

WINDOWEDX must be between 80 and 255 inclusive; WINDOWEDY between 25 and 255.

The problem here is that larger values actually have a dramatically different function: they set the target dimensions for each individual tile in pixels, rather than window dimension in tiles. To keep the launcher logical, limit it to changing the window size and leave tile-resizing to the ingame zoom function.

[Comment created by Pidgeot: 2015-02-17]
IMO, the fullscreen values should probably also be added.

I'm not sure about capping at 255; it wouldn't be possible to preserve any exact pixel settings that way.

missing png files for TWBT after installation of graphics packs

After installing a graphics pack, these files are missing and prevent TWBT to load a startup of dfhack:

• /data/art/transparent1px.png
• /data/art/white1px.png

[Issue created by McArcady: 2017-11-27]
[Last updated on bitbucket: 2017-12-01]

[Comment created by Pidgeot: 2017-12-01]
PR has been merged, so that should be fixed now.

Make update check non-blocking

Here a user describes:

If the device isn't connected [the PyLNP] seems to hang for about a min or two and then throws an error when the connection timeouts -- you can ignore the error and the pack works properly but the delay gets a bit annoying after awhile.

Adding an option to disable update checking would be a viable if inelegant fix - though still desirable from eg a privacy perspective.

Making the update check non-blocking would be a cleaner resolution of this issue. If an update was found, it could interrupt to pop up a notification; if no update was found or the check fails it passes silently.

[Issue created by PeridexisErrant: 2014-09-18]
[Last updated on bitbucket: 2014-09-19]

[Comment created by Pidgeot: 2014-09-19]
Perform update check in separate thread (fixes #25)

→ <<cset 8af740bcaa20>>

[Comment created by Pidgeot: 2014-09-18]
Agreed, this is definitely something that needs to be done.

Python3 download error and regexp problem

Error downloading <baseline>
Traceback (most recent call last):
  File "python-lnp\core\download.py", line 162, in __process_queue
    outfile.write(chunk)
TypeError: must be str, not bytes
Exception in thread Thread-2:
Traceback (most recent call last):
  File "python-lnp\core\download.py", line 162, in __process_queue
    outfile.write(chunk)
TypeError: must be str, not bytes

This is fixed for me by using outfile = os.fdopen(outhandle, 'wb') - but apparently the distinction is a windows-only thing, so I'm not going to simply add it to the pull.

I'm also having issues the version_text regexp. In Python3, the version_text is a bytes object - but the regexp is a string. If I use str(version_text), the expression Version: (.+) returns version_string with ' appended.

This is making it quite difficult to test my direct downloading branch! (which I suspect is not ready for merging yet).

[Issue created by PeridexisErrant: 2015-01-19]

[Comment created by PeridexisErrant: 2015-01-19]
Cool.

Any idea on the regexp issue? I've got pretty much everything else working now...

[Comment created by Pidgeot: 2015-01-19]
Checking the docs, 'wb' should be fine on all platforms, so go ahead and include it in your PR.

[Comment created by PeridexisErrant: 2015-01-19]
Ah, that's fixed it. Thanks!

[Comment created by Pidgeot: 2015-01-19]
Due to computer issues, I am not able to perform any testing at the moment, but for now, you should be able to call decode('utf-8') on the bytes. That won't be the final code, but it should work for testing.

Ideas for future work on Mods tab

If a manifest file is eventually added, it will be possible to do more with mods - and probably also make non-critical parts faster, but for most I value the correctness of file analysis over the speed of manifest reading. I dislike the idea of a manifest, as it's a unique requirement which undermines interoperability and simplicity, and may not always be correct.

The big one is of course using mods for old DF versions - it would have to work by using the old mod, old vanilla, and new vanilla to create a new mod file (by the current 3-merge logic), and then use that in a mod merge for the new version.

It would be possible to determine which DF version (of the available baselines) a mod is for, by checking how many lines of raws are changed relative to each and returning the least. This would eliminate the requirement for a manifest, but force manual download of the relevant baseline and in some cases would slow the whole process a considerable amount.

If multiple baselines are present, they could be stored in reduced format relative to the newest. I'm not sure if the uncommon disk space savings would be worth the runtime cost, but it's an idea.

TLDR; further enhancement is probably going to either force a manifest and thereby lose some elegance of the format, or may be too runtime-expensive to be worthwhile since the logic would have to run at least partially in many edge-ish cases such as multiple DF versions found, and could not store results (without... a manifest).

No action required, this just seemed a good place to dump my thinking that the mods tab is at a logical finishing point, and why.

[Issue created by PeridexisErrant: 2015-01-20]
[Last updated on bitbucket: 2015-03-02]

[Comment created by PeridexisErrant: 2015-01-29]
Another thing that would be hard but useful: handling mods with included graphics.

The current situation assumes mods are distinct from graphics packs, and then applies vanilla graphics packs to the final mix. Unfortunately this isn't always a great result; mod creatures especially will stand out due to the missing sprites (and use tiles instead).

Many mods actually do distribute alternative versions, which are integrated with various popular graphics packs. However, the PyLNP will disregard the image files, and for compatibility it's best to use the ASCII-graphics version of mods (as many graphics packs make other raw changes).

---

Update 2015-02-11: now handling graphics by a simple conditional copy. If the file doesn't exist, copy it and set status to 1. If it does, replace the old version and set status to 2.

[Comment created by PeridexisErrant: 2015-03-02]
Discussion has moved to issue #63

Open tileset image file from Graphics tab in LNP

To help select which tilesets I want to use, it would be handy if I could right-click open files straight from the list.

[Issue created by Former user: 2017-05-25]
[Last updated on bitbucket: 2018-01-06]

[Comment created by Pidgeot: 2018-01-06]
I'm going to decline this request, because although it is indeed not difficult to do, the "Open folder" button is likely much more useful as your OS file manager will do a better job of giving an overview of the available graphics.

(Longer-term, there's #58 for an internal way of dealing with previews, but no ETA on when I can get back to that.)

Unable to click any buttons in osx both with precompiled binary or running from source.

Exception:

#!python

$ python lnp.py
Note: PIL not found and Tk version too old for PNG support (8.5).Falling back to GIF images.
WARNING: LNP folder is missing!
Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 1470, in __call__
    return self.func(*args)
  File "/Users/cden/Downloads/Pidgeot-python-lnp-f585d2edeb01/tkgui.py", line 495, in <lambda>
    command=lambda: self.cycle_option('temperature'))
  File "/Users/cden/Downloads/Pidgeot-python-lnp-f585d2edeb01/tkgui.py", line 1044, in cycle_option
    self.lnp.cycle_option(field)
  File "lnp.py", line 290, in cycle_option
    self.save_params()
  File "lnp.py", line 73, in save_params
    self.settings.write_settings()
  File "/Users/cden/Downloads/Pidgeot-python-lnp-f585d2edeb01/settings.py", line 231, in write_settings
    self.write_file(filename, self.in_files[files])
  File "/Users/cden/Downloads/Pidgeot-python-lnp-f585d2edeb01/settings.py", line 262, in write_file
    self.field_names[field], self.settings[field]), text)
  File "/usr/local/Cellar/python/2.7.6_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/re.py", line 151, in sub
    return _compile(pattern, flags).sub(repl, string, count)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 4879: ordinal not in range(128)

[Issue created by cdenyar: 2014-08-09]
[Last updated on bitbucket: 2014-08-15]

[Comment created by Pidgeot: 2014-08-10]
Judging by the log above, that would be expected - the directory structure isn't in place, so there isn't a graphics folder, or any packs to install. Please note that the binaries here do not contain any of that; you need to supply that yourself.

The folder containing the main script/binary needs to contain a folder named LNP, with a subfolder Graphics where the graphics packs are located - please see the current Windows starter pack for the details (you can use the same files for graphics on OS X, only utilities are platform-specific).

I would still like to see your previous init.txt and d_init.txt files for testing purposes (it is a bug, but I can't fix it without being able to reproduce it).

[Comment created by cdenyar: 2014-08-10]
Fresh install worked. Missing icon at the top of application and Install Graphics button and Open Graphics Folder buttons don't do anything. Let me know if oyu want any more logs, new issue submitted, etc.

(have pillow installed via homebrew.)

[Comment created by Pidgeot: 2014-08-09]
It looks like the program is complaining about the contents of one of the configuration files, but I cannot reproduce this here.

Could you please attach your init.txt and d_init.txt? Also, could you please try with a fresh copy of Dwarf Fortress?

Finally, I would also like to know the output of python --version.

Color scheme selector

Make an interface to allow installation of color schemes, similar to graphic packs.

[Issue created by Pidgeot: 2014-07-18]
[Last updated on bitbucket: 2014-08-17]

[Comment created by Pidgeot: 2014-08-17]
Color scheme selector and preview (fixes #1)

→ <<cset 315754c902df>>

[Comment created by PeridexisErrant: 2014-08-17]
The Manilla DF launcher has one of these, and the (java) source is available on request. Probably not relevant, but I though it worth mentioning.

macOS: Support embedded Dwarf Fortress

I'm going to go forward with the plan to embed DF inside the PyLNP app bundle, and then copy it to the user's filesystem when the app is launched. This will work around Mac security restrictions.

Inside PyLNP.app/Contents/Resources/Game I will put the LNP and Dwarf Fortress x.yy.zz directories, as well as a version.txt file that contains a unique string for the starter pack release.

When PyLNP runs, it will read the version.txt file. Suppose it says 0.44.12-r02-mac, then PyLNP will be directed to look at ~/Library/Application\ Support/Dwarf\ Fortress/0.44.12-r02-mac.

If this directory does not exist yet, then the Game directory is recursively copied to ~/L/AS/DF/0.44.12-r02-mac.

In order to run a migration step, it will check for a symlink in ~/L/AS/DF/Previous to another DF directory. If this symlink exists, then it will offer to migrate saved games. After migration, it will offer to delete the old installation. The Previous symlink is updated to the new directory.

PyLNP then changes the current working directory to ~/L/AS/DF/0.44.12-r02-mac and continues.

---

Does that sound reasonable?

[Issue created by rgov: 2018-07-20]

[Comment created by rgov: 2018-07-20]
Downsides:

• If there are any files that PyLNP does not know how to automatically migrate, then the user will face data loss when we offer to delete the old directory.

• If the user declines to delete the old directory, then they may not know how to find it later. We can put a shortcut in the Folders menu.

• The DF inside the app bundle is going to take up space after the initial copy. Perhaps it could instead be a compressed tarball that gets recursively copied, which would save a bit of space. Or perhaps it could do something crazy with symlinks, but that doesn't seem reliable.

[macOS] Support Retina graphics

macOS has been high resolution (around ~144dpi) since ~2012, but for some reason PyLNP 0.13 is low resolution and therefore blurry.

I understand that it's a funny complaint considering we're talking about Dwarf Fortress.

[Issue created by rgov: 2018-07-02]
[Last updated on bitbucket: 2018-07-08]

[Comment created by rgov: 2018-07-02]
Relevant, with solution: https://stackoverflow.com/questions/40484659/pyinstaller-tkinter-window-low-resolution-in-app-bundle-but-not-in-app-program

[Comment created by rgov: 2018-07-02]
Patch submitted in pull request #77.

[Comment created by Pidgeot: 2018-07-08]
Assuming fixed since PR was merged. Feel free to re-open if the next release still has this problem.

Add `TWBT` as a printmode option

TwbT is now activated by setting [PRINTMODE:TWBT], so it would be good to have this as a printmode option. If TwbT is not available vanilla DF defaults to STANDARD, so it won't break anything.

[Issue created by PeridexisErrant: 2014-09-26]
[Last updated on bitbucket: 2014-10-11]

[Comment created by Pidgeot: 2014-10-11]
Massive refactoring work
Allow more flexible placement relative to DF directory
Add TWBT detection and print mode options (fixes #31)

→ <<cset 4daf6a30f367>>

[Comment created by Pidgeot: 2014-09-26]
I already have local changes ready to add this (and TWBT_LEGACY) as options if TWBT is found in the df/hack/plugins folder, but as far as I could tell from the TwbT forum thread, this was something that mifki might change fairly soon. Because of that, I won't be committing this until I make the next build.

Baseline download fails

Running from source, in the 40_23 starter pack (with manually-added baseline removed):

Error downloading http://www.bay12games.com/dwarves/df_40_23_win_s.zip
Traceback (most recent call last):
  File ".\python-lnp\core\download.py", line 160, in __process_queue
    total = response.info().getheader('Content-Length')
AttributeError: 'HTTPMessage' object has no attribute 'getheader'

Running the built v0.9 win32 version, everything works as expected.

[Issue created by PeridexisErrant: 2015-01-10]

[Comment created by Pidgeot: 2015-01-10]
Fix Python3 compatibility of download queue system (fixes #51)

→ <>

Error with command for "other automation" DFHack option

An error is displayed when enabling Other Automation plugins for DFHack via the launcher. This prevents the "burial" command automation from working correctly.

Setup:

• Enable "Other Automation plugins" option in LNP (DFHack Tab)
• Launch DF
• Load a world

Expected:

• No errors in console
• Burial plugin run every 7 days

Outcome:

• Error displayed (see below)
• Burial not run at all

---

When enabling the "Other Automation plugins" option for DFHack, this error is displayed:

Starting autonestbox.
...RFF~3\DWARFF~1\Dwarf Fortress 0.40.24\hack\lua\utils.lua:595: error: invalid arg: 3: timeunits
stack traceback:
        [C]: in function 'error'
        ...RFF~3\DWARFF~1\Dwarf Fortress 0.40.24\hack\lua\utils.lua:595: in function 'processArgs'
        ...\DWARFF~1\Dwarf Fortress 0.40.24\hack\scripts/repeat.lua:19: in main chunk
        (...tail calls...)
Fixed feeding timers for 0 citizens.

It appears to be caused by a lowercase letter in PyLNP_dfhack_onLoad.init - line 9

multicmd autonestbox start; repeat -time 7 -timeunits days -command burial -name burial

The repeat command is used again on line 13 with an uppercase "U" in -timeUnits and appears to work fine

multicmd fix/blood-del enable; repeat -time 1 -timeUnits months -command [ fix/feeding-timers ] -name feeding-timers; repeat -time 1 -timeUnits months -command [ fix/stuckdoors ] -name stuckdoors

Manually editing the line causes a further error, which is then resolved by adding square brackets around the -command parameter, as below:

multicmd autonestbox start; repeat -time 7 -timeUnits days -command [ burial ] -name burial

This line works correctly and launches with no further errors. Note, however, that toggling any option on the DFHack tab will cause this file to be regenerated, and the manual edits lost.

---

I tried to locate the location this line is stored within the source, but was unable to. Hopefully it's a simple fix.

[Issue created by JyeGuru: 2015-01-30]

[Comment created by JyeGuru: 2015-01-30]
headdesk So sorry. It's been a long week.

[Comment created by Pidgeot: 2015-01-30]
This is a problem with the configuration of the pack you're using, not PyLNP itself - pack authors provide their own commands for this; they're not built-in. (Look in PyLNP.json for the configuration.)

Judging by the partial paths in the error message, I'm assuming you're on Windows, meaning you're using @PeridexisErrant's pack - he's the one you need to talk to. He should be notified about the problem; if he doesn't respond, I recommend you make a post on the forum thread.

LNP fails to launch on Arch Linux

OS: Manjaro 17.0.5 Gellivara
Kernel: x86_64 Linux 4.9.53-1-MANJARO

#!

[user@host ~]$ lnp
LNP/utilities/dwarf_therapist/DwarfTherapist (No such file or directory)
WARNING: Can't load or change keybinds with missing baseline!
Traceback (most recent call last):
  File "./PyLNP", line 11, in <module>
  File "/home/user/.dwarffortress-lnp-git/python-lnp/core/lnp.py", line 119, in __init__
    self.initialize_ui()
  File "/home/user/.dwarffortress-lnp-git/python-lnp/core/lnp.py", line 212, in initialize_ui
    self.ui = TkGui()
  File "./tkgui/tkgui.py", line 182, in __init__
    self.create_tab(GraphicsTab, 'Graphics')
  File "./tkgui/tkgui.py", line 304, in create_tab
    tab = class_(self.n, pad=(4, 2))
  File "./tkgui/tab.py", line 30, in __init__
    self.read_data()
  File "./tkgui/graphics.py", line 36, in read_data
    self.read_graphics()
  File "./tkgui/graphics.py", line 200, in read_graphics
    packs = self.packs = [p[0] for p in graphics.read_graphics()]
  File "/home/user/.dwarffortress-lnp-git/python-lnp/core/graphics.py", line 79, in read_graphics
    return tuple(sorted(result, cmp=helpers.sort_underscore_first))
TypeError: 'cmp' is an invalid keyword argument for this function
[WARNING] startlnp: Failed to start PyLNP

[Issue created by Former user: 2017-10-25]
[Last updated on bitbucket: 2017-11-23]

[Comment created by Pidgeot: 2017-10-25]
It seems like there's a Python 3 bug in the current version - the binary was compiled with Python 3.

I don't know where you got this particular binary, but if you can tell the person who provided it to re-compile with Python 2, it should work. Alternatively, consider using the binaries from here until now.

I've fixed this locally, but it will probably take a few days before I can push it (due to other changes that need to be pushed at the same time, but aren't quite ready yet).

[Comment created by Pidgeot: 2017-11-23]
Fix Python 3 compatibility by converting cmp-style function to key-style function (fixes #149)

→ <<cset 1ff97beb109b>>

[OSX] Problems while testing the new (reduced raw) graphics installation

If I use the graphic packs as they were before I get this error:

Something went wrong: the graphics folder may be missing important files. Graphics may not be installed correctly.
See the output log for error details.
There's nothing in the std logs.
What happens in this case is that the launcher copies the /data folders, but not the /raw folders, so the installation is broken.

If I let the launcher simplify the graphics the process is overly aggressive and removes the raw/object folder, and doesn't checks for differences between the baselines and graphics.
I haven't tried manually creating the reduced raws yet.
If I try to install these wrong reduced raws I get the same error as above.
Tested with brand new 9.2 codebase.

[Issue created by fricy: 2015-02-06]
[Last updated on bitbucket: 2015-02-09]

[Comment created by PeridexisErrant: 2015-02-07]
Drat, I think the remaining problem is in copying a directory over. Manual walk and copy it is... tomorrow.

[Comment created by fricy: 2015-02-08]
Installation into df/raw is OK, but fails in save/region/raw.

Traceback (most recent call last):
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/core.graphics", line 91, in install_graphics
NameError:
global name 'FileNotFoundError' is not defined

Simplify button deletes complete /raw folder.
Testing Baseline downloads: The process empties the raw/interaction_examples folder, resulting in missing files

[Comment created by PeridexisErrant: 2015-02-08]
Yep, that's because I set the can_rebuild function to strict - if it can't rebuild the current raws perfectly, it won't replace them with an updated set. This means that your saves will never be broken, but if you make minor changes (or more importantly lack the installed_raws.txt as unmodded saves all do) they won't be updated.

[Comment created by Pidgeot: 2015-02-06]
Notifying @PeridexisErrant.

Given that this is only just uploaded, I'm pulling the binaries pending further investigation.

Did you experience the same issue with 0.9.1a, or have you not tested that? How about 0.9?

[Comment created by PeridexisErrant: 2015-02-07]
Yep, inserting that before each instance of copy_tree seems to have worked. Pull incoming.

[Comment created by fricy: 2015-02-08]
Cool, simplify output looks about right.
But suddenly I started getting: No savegames to update error.

[Comment created by fricy: 2015-02-08]

In core/graphics.py, replace except FileNotFoundError: with except:. There are two such instances.

Cool, that's a fix.

Simplify button deletes complete /raw folder.
Not sure about this one; I'd expect at least /raw/graphics to live on.

Reverting changes to baselines in commit 328 preserves raw/graphics, but that's not enough, we need to keep all differing files in raw/objects too. It was working in 0.9.0 (277), but rewritten since, so not sure what needs to be fixed.

I'm pretty sure DF doesn't use that folder at all, but to keep it, insert this on line 77 of core/baselines.py:

No idea if it's used at all, but I'd rather keep it in case something explodes. :)

And a different issue that stems mostly from an error I made, but I think is worth bringing up here:
Recently I made a mistake of playtesting Accelerated-Modest Mod and some files got left behind and distributed with vanilla raws in Macnewbie. The extra raw files are harmless in this case, unused material templates that are not used in any creatures, but are still needed by the world.sav generated by this version. Up to now this hasn't been a problem, because the launcher just overwrote raws with he same name, but didn't clean the folder of extra files, but since now we clean it, this will break those saves, and I think this issue will affect players with various modded saves too.

I think cleaning the (df/raw) folder is a good move, and should have been the standard all along. But -at least for the transition period - we should have a checking logic for extra files in save/region/raw, backup them up in case something goes wrong, and prompt the user to the issue, and tell them what to do if they cannot load the save.

[Comment created by PeridexisErrant: 2015-02-08]
http://pastebin.com/M7GdP6kQ

This is not happening for now, though I didn't test much before. Changes include

Explicitly list patterns to keep when simplifying. Compare file contents, not with filecmp. Test can_rebuild in strict mode. Remove FileNotFoundError in mods.py. Coerce colors to ints, since we're processing them.

The main things to use read() instead of filecmp and test string equality; and explicitly list patterns to keep for each raw type in the baselines simplify function.

Refactoring the function to update raw graphics now always uses the mods merge logic, so if it aborts you can check the contents of baselines/temp/ to see what's happened.

Edit: Ommission of raw/objects was accidental, and fixed when I made the simplify function more explicit.

[Comment created by fricy: 2015-02-07]
Testing: Behaves exactly like 9.1a(325):

At least I found where the logs are.
Note: Output is different for ascii and graphics set installation.

Traceback (most recent call last):
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/core.graphics", line 65, in install_graphics
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/core.graphics", line 239, in update_graphics_raws
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/distutils.dir_util", line 163, in copy_tree
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/distutils.dir_util", line 163, in copy_tree
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/distutils.dir_util", line 139, in copy_tree
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/distutils.dir_util", line 76, in mkpath
distutils.errors
DistutilsFileError
**could not create '/var/folders/q6/39ysly095pv2rwn1v8gvcn500000gn/T/tmpPDAYjt/graphics/afro': No such file or directory
**

Traceback (most recent call last):
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/core.graphics", line 65, in install_graphics
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/core.graphics", line 239, in update_graphics_raws
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/distutils.dir_util", line 163, in copy_tree
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/distutils.dir_util", line 163, in copy_tree
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/distutils.dir_util", line 167, in copy_tree
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/distutils.file_util", line 156, in copy_file
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/distutils.file_util", line 44, in _copy_file_contents
IOError
**[Errno 2] No such file or directory: u'/var/folders/q6/39ysly095pv2rwn1v8gvcn500000gn/T/tmp_vTwo7/graphics/example/dwarves.bmp'
**

[Comment created by Pidgeot: 2015-02-08]
In that case, it seems to "just" be because @PeridexisErrant chose to omit /raw/objects for graphics packs - since I'm not sure exactly why he did that, I don't want to revert it until I hear from him.

[Comment created by fricy: 2015-02-08]
http://pastebin.com/H2vnUyiM

It should have left :

• creature_subterranean.txt
• inorganic_stone_gem.txt
• inorganic_stone_layer.txt
• inorganic_stone_mineral.txt
  reaction_other.txt

[Comment created by PeridexisErrant: 2015-02-07]
Try https://bitbucket.org/PeridexisErrant/python-lnp/commits/d37ec0d7c09a8368c2ca1349bbae9c9b2eb1034c

and let me know how it goes - I'll try to test more later tonight or tomorrow.

[Comment created by fricy: 2015-02-06]
Ok, checking version features with my own compiles:
8.b: (239)
Does'nt need the baselines folder,
detects reduced raw graphics, (suprising...) - installs them incorrectly
full graphic raws are installed correctly
Running simplify graphics produces error messages, logs are empty.
9.0 (277):
full graphic raws are installed correctly
Needs baselines
simplify seems to work correctly, but produces an error message (No files in: )
simplified raws are installed incorrectly
-» it looks to me that baseline/temp/raw is assembled correctly, but doesn't get copied to df/raws and saves/region/raws.
9.1a(325):
Doesn't install graphics at all (reduced or full)
(Something went wrong: the graphics folder may be missing important files. Graphics may not be installed correctly.
See the output log for error details.)
Simplifiy deletes the whole raw/objects folder!
9.2 (342):
Installing full or reduced raws produces the same error message as in 9.1a. Installation incorrect, only the data folder gets copied, raw doesn't
Simplify delets the whole folder, but the error message (in 9.0) is fixed

This test was done without messing with mods, only testing the graphics installation

[Comment created by Pidgeot: 2015-02-09]
I will merge the pull request now and get the builds up within the next hour or so; any further stuff is probably a separate issue anyway. Good work, PE :)

[Comment created by fricy: 2015-02-09]
I say it's good to go!
Did some testing with mod merging, and the save survived a few changes, so good job so far.
I'll try find some time to analyze merged mods, to see how well it does it job. I suppose you don't do anything fancy, I was thinking that graphics merging can be made more foolproof by only merging lines that contain the necessary strings, there are a few variants of TILE and GROWTH_PRINT.

[Comment created by Pidgeot: 2015-02-08]

That code wouldn't work because it only accepts one parameter, and there's two now. My hunch is that the bug is in the function that evaluates if a file should be removed or not, but I can't spot anything to base this on, so whatever.

No, it'll work just fine - I'm talking about replacing the function in graphics.py, not in baselines.py.

[Comment created by PeridexisErrant: 2015-02-07]
Turns out that completely reworking the graphics code was a little more complex than I thought, and I misunderstood what happened in a couple of cases with tempdirs. I've got a fix that at least shows no errors and returns the right things, but I'm going to do a little more testing before pushing it...

[Comment created by PeridexisErrant: 2015-02-09]
Ah, that's it.

I've decided that for the sake of speed and avoiding endless false-negatives to revise the can_rebuild function. Instead of rebuilding and comparing the whole raws, it now simply confirms that all the components to do so are available. This works, is save-safe for things which are logged correctly, skips unlogged saves, and only fails if a log is present and incorrect - or something has the same name but incompatible content.

So long as mods change the name between incompatible versions (ie use a version number!), this should work.

[Comment created by fricy: 2015-02-08]
Nope, still doesn'r want to update the saves. And simplify:

Exception in Tkinter callback
Traceback (most recent call last):
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/Tkinter", line 1532, in call
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/tkgui.graphics", line 255, in simplify_graphics
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/core.graphics", line 213, in simplify_pack
File "/Users/fricy/Github/python-lnp/build/lnp/out00-PYZ.pyz/core.baselines", line 129, in remove_vanilla_raws_from_pack
TypeError: file() takes at most 3 arguments (4 given)

[Comment created by Pidgeot: 2015-02-08]
baselines.py needs a from io import open in order for the changes to work in Python 2.

[Comment created by Pidgeot: 2015-02-08]

Cool, that's a fix.

Excellent, just pushed that change.

Simplify button deletes complete /raw folder.

Not sure about this one; I'd expect at least /raw/graphics to live on.

Reverting changes to baselines in commit 328 preserves raw/graphics, but that's not enough, we need to keep all differing files in raw/objects too. It was working in 0.9.0 (277), but rewritten since, so not sure what needs to be fixed.

Wait, 328 only changed stuff related to counting the number of files changed. How the hell does that prevent raw/graphics from being preserved? Exactly which revision did you revert to?

FWIW: If all the files are identical, then it's expected that the raw folder would be deleted for the pack.

If you're sure they're not identical, go back to 328 (+ pull request) for a moment, then change graphics.simplify_pack to this:

def simplify_pack(pack):
    """Removes unnecessary files from one graphics pack."""
    print("Before simplification of ",pack)
    for x,y,z in os.walk(paths.get('graphics', pack)):
        print(x,z)
    a = baselines.simplify_pack(pack, 'graphics')
    print("After initial simplification of ",pack)
    for x,y,z in os.walk(paths.get('graphics', pack)):
        print(x,z)
    b = baselines.remove_vanilla_raws_from_pack(pack, 'graphics')
    print("After removing vanilla ",pack)
    for x,y,z in os.walk(paths.get('graphics', pack)):
        print(x,z)
    c = baselines.remove_empty_dirs(pack, 'graphics')
    print("After removing empty dirs ",pack)
    for x,y,z in os.walk(paths.get('graphics', pack)):
        print(x,z)
    if not all(isinstance(n, int) for n in (a, b, c)):
        return False
    return a + b + c

Run simplification and post the results on pastebin or something (testing on one pack should be sufficient, as long as the problem manifests itself for that pack). Hopefully it'll point in the right direction. (For good measure, mention which DF version you're testing with - I'm assuming 40.24, but I'd like to be completely certain).

My gut feeling is to do this:

• Merge the pull request.
• Change simplification to always preserve the entire raw directory.
• Fix whatever issue is causing the raw folder to be deleted (unless it's intentional!)

Unfortunately, I am rapidly running out of time if this is to be resolved before my CPU goes in for replacement - I won't be able to make builds until it's returned, so unless this has to wait another week or so to get fixed, my best chance is during the next 8-9 hours or so. @PeridexisErrant: I realize that it will be Monday morning by the time you read this, so you might not have time to code up a fix yourself. I think I can squeeze it the coding myself, but I'd need your input ASAP to be absolutely certain that my plan won't lead to a problem.

[Comment created by fricy: 2015-02-08]
I must have made some mistake yesterday, because I can't reproduce the behavior when the simplify command removes raw/graphics, so disregard my comments about reverting 328.
raw/objects are still removed as reported, that function only works correctly in 0.9.0 (commit 277).

FWIW: If all the files are identical, then it's expected that the raw folder would be deleted for the pack.

Yeah, I'm aware of this, but everything get's deleted.

(For good measure, mention which DF version you're testing with - I'm assuming 40.24, but I'd like to be completely certain).

Correct.

Run simplification and post the results on pastebin or something (testing on one pack should be sufficient, as long as the problem manifests itself for that pack).

That code wouldn't work because it only accepts one parameter, and there's two now.
My hunch is that the bug is in the function that evaluates if a file should be removed or not, but I can't spot anything to base this on, so whatever.

Unfortunately, I am rapidly running out of time if this is to be resolved before my CPU goes in for replacement - I won't be able to make builds until it's returned

Nah, it's perfectly ok to wait a week, I'd rather release something that's tested properly. We are not EA after all... :)

[Comment created by PeridexisErrant: 2015-02-08]
Ah, I think that's a related issue - the newline was either missing, and causing problems, or present - and thus being included in the path. I've refactored a little and fixed that.

[Comment created by Pidgeot: 2015-02-08]

global name 'FileNotFoundError' is not defined

In core/graphics.py, replace except FileNotFoundError: with except:. There are two such instances. (FileNotFoundError is new to Python 3; not sure how I missed that one.)

Simplify button deletes complete /raw folder.

Not sure about this one; I'd expect at least /raw/graphics to live on. @PeridexisErrant?

Testing Baseline downloads: The process empties the raw/interaction_examples folder, resulting in missing files

I'm pretty sure DF doesn't use that folder at all, but to keep it, insert this on line 77 of core/baselines.py:

#!python
if folder == 'baselines':
    keep += [os.path.join('raw', 'interaction examples', '*')]

Sidenote: core/baselines.py:82 is missing a comma between 'init' and '*'. I believe it still works as intended because the * globs the path separator as well, but it should probably still be fixed.

[Comment created by Pidgeot: 2015-02-06]
Oh dear god, apparently something's going horribly wrong with the OS X packaging. There are probably many builds that are wrong; I can't tell when that screenshot is from.

Please compare to your own builds for now, I'll be fixing it here in the meantime. (EDIT: Should be fixed for 0.9.2, but I'll keep a close eye on it)

[Comment created by fricy: 2015-02-06]
Hmm, there's something off, and I don't know what.
I downloaded your 0.9.1a and compared it with a self built 0.9.1a

Your 9.1a doesn't exhibit this behavior, but looks a bit different, doesn't have the mods tab, the selection colors are red, not green, and most importantly doesn't prompt me for the baseline folder.

http://i.imgur.com/n1PDobz.png

[Comment created by Pidgeot: 2015-02-07]
This seems relevant: http://stackoverflow.com/questions/9160227/dir-util-copy-tree-fails-after-shutil-rmtree - but I don't have time to do testing right now.

@fricy, if you find time before either me or PE, can you try adding the line dir_util._path_created = {} to update_graphics_raws in graphics.py? Put it just before tmp = tempfile.mkdtemp(). If that doesn't do it, try putting it before each copy_tree call.

[Comment created by fricy: 2015-02-08]
I like that, but something is broken with it, won't change the graphics on a newly generated world.

Also: a new line character is missing from the installed_raws.txt generation between graphics and mods resulting in a corrupting configuration.

Observe: select ascii. Install any 2 mods. check installed_raws.txt. install another graphics. check again.