Crash from somewhere with dragonfly behind RefineryCMS

[markevans]

Crash from somewhere with dragonfly behind RefineryCMS

[markevans]

looks like an rmagick problem - what version of rmagick are you using?
Version 2.12.2 works ok for me but 2.13 has been known to segfault

[plukevdh]

looks like the only one I have installed is rmagick (2.12.2)

[parndt]

I've found that ruby 1.9.2 is quite prone to crashing with rmagick / dragonfly when creating images. It's a confusing issue when all you get's a segfault. :/

[markevans]

Yes I find it infuriating. RMagick works well for the most part but when it segfaults like that there seems to be nothing you can do (other than try different versions of rmagick/imagemagick).
I've wondered if at some stage I might rewrite the default processors to shell out to the command line and use imagemagick. (this can already be done with custom processors btw)

[plukevdh]

There are a few libraries doing this already (http://github.com/probablycorey/mini_magick, http://github.com/konung/quick_magick) Maybe we should look into implementing dragonfly around one of these libs instead?,

[parndt]

I would support a move to shelling out as an alternative processor to RMagick. Most of the support queries we get around images / files are to do with installing RMagick in some way or another.

[plukevdh]

I'm up for helping with some of that too I think. I'd recommend mini_magick over quick (unless you are looking to write from scratch). Mini has more community support behind it and appears better maintained.

[parndt]

Even just a choice of processors would be useful, though I know how much complexity this can add to testing.

[fabn]

Same happens to me, if I use 1.9.2 it segfaults at every image processing function, but if I switch to 1.8.7 it works.

Configs and stacktrace: http://gist.github.com/615075

[htp]

Tossing my hat into the ring as well- I get a segfault using 1.9.2 and every RMagick newer than and including 2.12.1. (I haven't tried older versions.)

Info and stacktrace at http://gist.github.com/626800

Out of curiosity, what versions of ImageMagick is everyone using? I'm using 6.6.4-5, installed with Homebrew on Snow Leopard.

[plukevdh]

Whatever homebrew installs, last update was 6.6.4-2, so that I guess. Didn't work then.

[htp]

Welp, just tried ImageMagick 6.6.3-4, 6.6.3-8, 6.6.3-9, and 6.6.4-2 with RMagick 2.12.2 (reinstalled the gem for each ImageMagick) to try to reproduce markevans' working environment from above, but no dice.

I can confirm fab's suggestion of switching to ruby 1.8.7 (and using the latest ImageMagick and RMagick) works, but I'd rather make things work for 1.9.2.

If someone could provide me some guidance to jumping out to the shell, I'd be up for taking a stab at a new processor.

[plukevdh]

checkout http://github.com/probablycorey/mini_magick. Probably the easiest way to go in the way of transitioning.

[markevans]

although something like mini_magick or quick_magick would work well, I don't think relying on another gem and adding an extra layer is necessary, because shelling out to the command line is pretty easy anyway.

so I've just started doing some stuff in a branch http://github.com/markevans/dragonfly/tree/imagemagick
which adds an ImagemagickProcessor. feel free to comment/fork/etc.

@htp in the meantime you can easily make custom processors using something like

Dragonfly[:images].processor.add :my_custom_process do |temp_object, *args|
  tempfile = Tempfile.new('dragonfly')
  tempfile.binmode
  tempfile.close
  `convert -some options #{temp_object.path} #{tempfile.path}`
  tempfile
end

to convert to a different format, you can do, e.g. Tempfile.new(['dragonfly', '.jpg']) instead,
and return e.g. [tempfile, {:format => :jpg}] rather than just tempfile (though the correct place to encode is in a custom encoder - see the docs)

[parndt]

Seems rmagick needs a new maintainer, wonder if it's time for a giant joint effort?

[markevans]

do you mean to step up and maintain rmagick? or to move away from it in Dragonfly?

[parndt]

Well if we had a go and patched the issues we had then we may not need to move away. It'd be good to gauge interest in either option

[markevans]

yeh that's true.

though having thought a bit more about the moving away to the command line option, I do like the idea of not having to depend on the extra gem, which would make it especially easier for Windows/Jruby users as I know there are one or two issues with these too.

Another thing is, it would get rid of the memory issues some people are having.

Paperclip uses the command line and doesn't seem to have any problems with it.

I don't know about anyone else, but I wouldn't have the time (or inclination really) to help with maintaining RMagick, not least because there's a load of c code in there.

Hats off to the guys who have been doing it up till now though

[parndt]

I would support a move to eliminate rmagick as long as the shelling out approach works Cross-OS / ruby runtime. :-)

[htp]

Hey all,

I took markevans' start on the ImageMagick processor and ran with it as best I could. (Getting up to speed with ImageMagick itself and RSpec took some time.)

Anyway, I copypasta'd the RMagick tests to create ImageMagick ones, and made some changes to pass all of the crop and resize_and_crop tests. I also renamed the processor and utils from ImagemagickFoo to ImageMagickFoo.

All of the changes are at https://github.com/htp/dragonfly ; I'd really appreciate another set of eyes (or more!) on it. If/when it's deemed ready, I'll submit a pull request and hopefully we can close this issue.

[htp]

Whoops, spoke too soon. Looks like the above's just a first step; I've got a bunch of other modules to take care of before we can think of closing this.

[htp]

Okay, reporting back in.

I've got an ImageMagick analyser, encoder, generator, and processor built out, along with a Rails config to replace the RMagick one. The individual specs all pass, but I'm having some trouble getting rake spec to play nicely. (We might want to update the rails:setup task, too- it doesn't work for Rails 3 for me.)

So far, the new ImageMagick stuff is working nicely with an in-house Rails 3 app (Ruby 1.9.2 and ImageMagick 6.6.4-5 on Snow Leopard). Anyone want to help me kick the tires?

[parndt]

I'll hook your fork into Refinery CMS today and see if our specs and features still all pass and the application still works with RMagick completely uninstalled from my system.

Thanks for all your effort on this!

[parndt]

Works like a dream on my system, I've actually changed Refinery CMS over to use your code (but on our fork so that it's safe for you to commit to your repository!):

resolve/refinerycms@d26b97f

The only issue we have is when we run our tests which upload a .txt file as an image (to check that you can't do that) we get these in the console as it hits that test:

........................................................................-----........................-----.....................................................................................................
identify: no decode delegate for this image format '/var/folders/79/79oz3ax0FYGpAs7GE3sDYU+++TI/-Tmp-/RackMultipart20101111-5915-sf0fjr-0' @ error/constitute.c/ReadImage/533.
identify: no decode delegate for this image format '/var/folders/79/79oz3ax0FYGpAs7GE3sDYU+++TI/-Tmp-/RackMultipart20101111-5915-sf0fjr-0' @ error/constitute.c/ReadImage/533.
identify: no decode delegate for this image format '/var/folders/79/79oz3ax0FYGpAs7GE3sDYU+++TI/-Tmp-/RackMultipart20101111-5915-sf0fjr-0' @ error/constitute.c/ReadImage/533.
identify: no decode delegate for this image format '/var/folders/79/79oz3ax0FYGpAs7GE3sDYU+++TI/-Tmp-/RackMultipart20101111-5915-sf0fjr-0' @ error/constitute.c/ReadImage/533.
......

RMagick didn't produce these, any idea how we can suppress them?

Thanks again, so amazing!

[htp]

Woah hey, Refinery! I've used your guys' stuff for a few projects before- glad I could finally give back! :)

I remember seeing those messages while I was working; I decided to ignore 'em at the time because the specs were passing and I wanted to kick something functional out into the world before going "full-perfectionist" on it.

That's on my list of things to clean up, though (I've got a few ideas to try)- I'll also update the docs and scrub some not-so-DRY code to make it easier for markevans to pull this into dragonfly proper.

Thanks again for taking a look!

[htp]

Okay, docs updated, rake spec running again, and that the stderr messages from the ImageMagickAnalyser test are now suppressed.

[parndt]

@ htp - we had an issue created related to Windows on our issues tracker:

https://github.com/resolve/refinerycms/issues#issue/282

(we're using your code in our refinerycms fork, still).

Loving no longer using rmagick. Nice work.

[julesce]

Hi guys, I just pulled the latest version of refinery and am still experiencing the error mentioned above by Parndt. I'm still quite new to Ruby, and am trying to figure out the issue, but please let me know if I can provide any more details to assist with the debugging. Thanks :)

[julesce]

For what it is worth, I decided to try Dragonfly on my system seperately from Refinery in an effort to narrow things down.

I used the 0.7.7 gem with a new Rails 3.01 app (Ruby 1.9.2 on Windows 64bit) and it is working fine.

Not sure how much this helps though, since it seems that the version of Dragonfly that Refinery is using at the moment is the brand new one that throws out rmagick?

I suppose the next step is to try out the latest source code seperate from Refinery, will keep you posted...

[julesce]

Ok, my next step was to try out a simple rails 3.01 app with the refinery fork of dragonfly. All seemed to work fine there as well (brief testing).

So I decided to go back to the 0.9.8.5 tag of Refinery. And lo and behold, I am still experiencing the same error when uploading both images and resources (files):

Errno::ENOENT in Admin/imagesController#create
No such file or directory - file -b --mime -

dragonfly (0.7.7) lib/dragonfly/analysis/file_command_analyser.rb:17:in `popen'
dragonfly (0.7.7) lib/dragonfly/analysis/file_command_analyser.rb:17:in `mime_type'
dragonfly (0.7.7) lib/dragonfly/function_manager.rb:37:in `call'

This would point to some integration issue between Refinery and Dragonfly right?

[julesce]

I think I am finally onto something... The FileCommandAnalyser is being used, but since I am on a Windows machine the 'file' command is not getting much love... :P

So what would the work around for this be for Windows users, if there is one yet? :)

[plukevdh]

Use linux :)

[htp]

Hrm. I don't have a Windows install at hand to test against- can you try changing it to use the ImageMagickAnalyser instead and letting us know if that helps? (That'll be in lib/dragonfly/config/rails.rb, on line 11.)

[julesce]

@htp Thanks for that pointer. I updated the file and was then met with an error relating to validation of the mime type, found in the relevant model in Refinery. So I commented that line out and am now at the point where the files are being saved successfully :)

The only problem now is that the thumbnail generation is failing:

Command failed (convert -resize '213x135' C:/Users/James/AppData/Local/Temp/dragonfly20101115-3860-1c5u300 C:/Users/James/AppData/Local/Temp/dragonfly20101115-3860-k12knt) with exit status 1

Any thoughts on the next step? Is there is a specific version of Imagemagick that needs to be running with this latest version of Dragonfly? FWIW I am using 6.5.6 Q8.

And yes, I know you hate us Windows users :)

[markevans]

OK so the new version of dragonfly is up (0.8.0), which shells out to the command line by default - RMagick stuff is still in there but only optional, so users only need imagemagick to be installed now.

Thanks to htp and parndt for your great contribution!

[julesce]

If I wanted to fork Dragonfly to work with Windows 'out of the box', would I be correct in saying that I would look to update the FileCommandAnalyser so that it would be able to determine the mime type in either Windows or *nix?

Are there any other places that rely on *nix commands?

Thanks :)

[markevans]

@julesce - I think the FileCommandAnalyser is the only place where a specifically *nix command is relied upon. However it's only used if you explicitly analyse the mime_type of something (e.g. in a validation, etc.).

Having had a quick look at refinerycms - it appears as though it has an image_mime_type column on images, which will automatically call mime_type in order to set it.

I don't know why convert is failing for you though - that should work on windows I think

[mclosson]

I was able to resolve this issue on Windows by copying the file.exe command and its dependant shared libraries from a cygwin installation into the appropriate locations on my windows system.

Copy these files into "%systemroot%\system32" (c:\windows\system32 for me) from your /bin directory in cygwin
*cygmagic-1.dll
*cygwin1.dll
*cygz.dll
*file.exe

Next create the directory for the "magic" files:
mkdir c:\usr\share\file

And copy these files from your /usr/share/file directory in cygwin into c:\usr\share\file
*magic
*magic.mgc
*magic.mime
*magic.mime.mgc

That's it. File and Image uploads will now work fine with the current codebase of RefineryCMS!