maik/xml-simple

Easy API for working with XML documents

xml-simple's Issues

Release 1.1.9 to RubyGems?

Hi! I was wondering if you guys were planning on releasing 1.1.9 to RubyGems? Latest version is 1.1.8. Thanks!

is_a? instead of instance_of?

Hi,

Current XmlSimple use instance_of? to check class in every point.
I prefer is_a? to instance_of?, because is_a? returns true with subclass of target class.
The instance_of? returns true with only target class.
This limitation often require conversion from subclass of Hash into Hash.
For example, Mongoid use BSON::OrderedHash, which inherit Hash, to represent hash of models.
So conversion is required when using XmlSimple#xml_out.
What do you think?

Thanks.

Namespace awareness

XML Namespaces are ignored.

Docs currently failing to build on rubydoc.info

https://www.rubydoc.info/gems/xml-simple/1.1.5

spins and spins

Funky Output wth Camel Case Variables

Hello,

This piece of code is providing some funky output when dealing with Camel Case variables, tested with xml-simple v1.1.5. Where "content" is not correctly parsed still printed out as, Ruby Hash, although if you change "content" variable to something like "contentCamelCasedTest" the problem does not appear anymore. Also, if removed other variables that start with "content" plus Camel Case values, the problem is also avoided.

Please consider:

require 'xmlsimple'

test = {
  'subClientProperties' => {
    'subClientEntity' => {
      'appName' => 'app_name',
      'backupsetName' => 'backup_set_name',
      'clientName' => 'cv_clientname',
      'instanceName' => 'DefaultInstanceName',
      'subclientName' => 'subclient_name',
    },
    'commonProperties' => {
      'enableBackup' => 'true'
    },
    'content' => {
      'path' => [
        '/var/tmp',
        '/var/lib/test'
      ]
    },
  }
}

xs = XmlSimple.new(
  { 'ForceArray' => [ "path" ],
    'KeepRoot' => false,
    'NoEscape' => true,
    'RootName' => "App_CreateSubClientRequest",
    'AttrPrefix' => true,
  }
)

puts xs.xml_out test

Output:

irb(main):033:0* puts xs.xml_out test
<App_CreateSubClientRequest>
  <subClientProperties>{"path"=>["/var/tmp", "/var/lib/test"]}<subClientEntity>
      <appName>app_name</appName>
      <backupsetName>backup_set_name</backupsetName>
      <clientName>cv_clientname</clientName>
      <instanceName>DefaultInstanceName</instanceName>
      <subclientName>subclient_name</subclientName>
    </subClientEntity>
    <commonProperties>
      <enableBackup>true</enableBackup>
    </commonProperties>
  </subClientProperties>
</App_CreateSubClientRequest>
=> nil

The current version seems to be 1.9 unfriendly

Enumarator is a 2.0 data type.

What's the last 1.9 release?

Inconsistent behaviour when 'content' key contains an array

The issue:

XmlSimple.xml_in '<a>1<b>3</b>5</a>' 
# => {"b"=>["3"], "content"=>["1", "5"]}
XmlSimple.xml_out XmlSimple.xml_in '<a>1<b>3</b>5</a>' 
# => "<opt>[&quot;1&quot;, &quot;5&quot;]<b>3</b>\n</opt>\n"

or simply:

XmlSimple.xml_out({a: [{ 'content' => ['123'] }]})
# => "<opt>\n  <a>[&quot;123&quot;]</a>\n</opt>\n"

but all is ok when content key contains a string:

XmlSimple.xml_out({a: [{ 'content' => '123' }]})
# => "<opt>\n  <a>123</a>\n</opt>\n"

PS: Thanks for the library, it is awesome!

Add a tag for xml-simple-1.1.4

Can you add a tag for it please?

Problem with Keyconversion

Hallo, I try to convert a XML-String

<d:propfind xmlns:d="DAV:" xmlns:cs="http://calendarserver.org/ns/">
  <d:prop>
     <d:displayname />
     <cs:getctag />
  </d:prop>
</d:propfind>

but XmlSimple.xml_in(xml,{'KeepRoot'=>true}) convert this to:

{"propfind"=>[
  {"xmlns:d"=>"DAV:", "xmlns:cs"=>"http://calendarserver.org/ns/",
  "prop"=>[{
     "displayname"=>[{}],
     "getctag"=>[{}]
    }]
  }
]}

The d:propfind, d:prop, d:displayname and cs:getctag incorrect converted to String.
Is a option for that or is it a bug?

Danke

xml_in not working properly

 hash = XmlSimple.xml_in('<test><name>LE</name><value class="string">0.0</value></test>', {KeepRoot: true, ForceArray: false})
=> {"test"=>{"name"=>"LE", "value"=>{"class"=>"string", "content"=>"0.0"}}}

XmlSimple.xml_out(hash, KeepRoot: true)
=> "<test name=\"LE\">\n  <value class=\"string\">0.0</value>\n</test>\n"

Feature request - (does this make sense to you?)

Dear Maik,

I was wondering if you could add a feature to xml_out and xml_in where: when it produces xml from a Hash it sets a 'ruby_class' attribute to the class of the data element for the key or array element key for each element, and casting the text to that ruby_class when producing the Ruby Hash/Array?

Does such a feature make sense to you? If not, what are your thoughts?

Thanks in advance, (great gem btw)

G. Gibson
mistergibson

XmlSimple.xml_in seems to drop the first element of an XML tag

When I use XmlSimple v1.1.5, I import an XML file like the one below, and it seems to drop the first element of the XML tag as seen in the Hash output towards the bottom. Is there some option I'm missing?

λ cat search_by_change_id.xml
<inv:SearchByChangeRequestID
  xmlns:inv="http://Remedy_Change_Request_LIB/Invoke"
  xmlns:wser="http://Remedy_Change_Request_LIB">
  <request>
    <ChangeRequestID>CRQ000000905650</ChangeRequestID>
  </request>
</inv:SearchByChangeRequestID>

irb(main):008:0> require 'xmlsimple'
=> true
irb(main):009:0> hash = XmlSimple.xml_in(File.open("search_by_change_id.xml"))
=> {"xmlns:inv"=>"http://Remedy_Change_Request_LIB/Invoke", "xmlns:wser"=>"http://Remedy_Change_Request_LIB", "request"=>[{"ChangeRequestID"=>["CRQ000000905650"]}]}
irb(main):010:0> pp hash
{"xmlns:inv"=>"http://Remedy_Change_Request_LIB/Invoke",
 "xmlns:wser"=>"http://Remedy_Change_Request_LIB",
 "request"=>[{"ChangeRequestID"=>["CRQ000000905650"]}]}
=> {"xmlns:inv"=>"http://Remedy_Change_Request_LIB/Invoke", "xmlns:wser"=>"http://Remedy_Change_Request_LIB", "request"=>[{"ChangeRequestID"=>["CRQ000000905650"]}]}
irb(main):011:0>

NormaliseSpace not always sufficient to prevent whitespace making equivalent docs appear different

When using this library within vagrant-libvirt in order to help compare XML domain definition sent to libvirt vs what was actually applied by libvirt I discovered that docs that contain different whitespace will sometimes appear not to be equivalent even when using the option NormaliseSpace set to 2, due to the resulting structure may contain a content element while the other doc that did not have the extraneous whitespace will not contain this field at all.

e.g.

<domain xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' type=''>
  <name/>
  <title/>
  <description/>
  <uuid/>
  <memory/>
  <vcpu>1</vcpu>
  <cpu check="none" mode="host-model">


  </cpu>
  <os>
    <type>hvm</type>
    <kernel/>
    <initrd/>
    <cmdline/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <devices>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <input bus='ps2' type='mouse'/>
    <graphics autoport='yes' keymap='en-us' listen='127.0.0.1' port='-1' type='vnc'/>
    <video>
      <model heads='1' type='cirrus' vram='16384'/>
    </video>
  </devices>
</domain>

vs:

<domain xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' type=''>
  <name/>
  <title/>
  <description/>
  <uuid/>
  <memory/>
  <vcpu>1</vcpu>
  <cpu check="none" mode="host-model"/>
  <os>
    <type>hvm</type>
    <kernel/>
    <initrd/>
    <cmdline/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <devices>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <input bus='ps2' type='mouse'/>
    <graphics autoport='yes' keymap='en-us' listen='127.0.0.1' port='-1' type='vnc'/>
    <video>
      <model heads='1' type='cirrus' vram='16384'/>
    </video>
  </devices>
</domain>

require 'xmlsimple'

doc1 = <<EOF
<domain xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' type=''>
  <name/>
  <title/>
  <description/>
  <uuid/>
  <memory/>
  <vcpu>1</vcpu>
  <cpu check="none" mode="host-model">


  </cpu>
  <os>
    <type>hvm</type>
    <kernel/>
    <initrd/>
    <cmdline/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <devices>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <input bus='ps2' type='mouse'/>
    <graphics autoport='yes' keymap='en-us' listen='127.0.0.1' port='-1' type='vnc'/>
    <video>
      <model heads='1' type='cirrus' vram='16384'/>
    </video>
  </devices>
</domain>
EOF

doc2 = <<EOF
<domain xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' type=''>
  <name/>
  <title/>
  <description/>
  <uuid/>
  <memory/>
  <vcpu>1</vcpu>
  <cpu check="none" mode="host-model"/>
  <os>
    <type>hvm</type>
    <kernel/>
    <initrd/>
    <cmdline/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <devices>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <input bus='ps2' type='mouse'/>
    <graphics autoport='yes' keymap='en-us' listen='127.0.0.1' port='-1' type='vnc'/>
    <video>
      <model heads='1' type='cirrus' vram='16384'/>
    </video>
  </devices>
</domain>
EOF

xml1 = XmlSimple.xml_in(doc1, {'NormalizeSpace' => 2})
xml2 = XmlSimple.xml_in(doc2, {'NormalizeSpace' => 2})
xml1 == xml2

will result in false.

I've added some code to vagrant-libvirt to walk the returned structure and remove the content if it's empty since if the whitespace is not relevant there should be no need for the attribute to be defined.
https://github.com/vagrant-libvirt/vagrant-libvirt/blob/439830b6d42758e2607c84d518f61be4235a8213/lib/vagrant-libvirt/util/xml.rb#L25-L43

If I then output the xml1 and input it again, it will have dropped the content field, so I'm wondering if maybe it should be not set at all in the case NormaliseSpace is 2 and after stripping it is empty?

Failing testcase

This testcase fails here:

Try parsing default external file (scriptname.xml in script directory)

opt = XmlSimple.xml_in(nil, { 'force_array' => false, 'key_attr' => %w(name key id) })
assert_equal({ 'location' => 'tc_perl_in.xml' }, opt)

1. Error:
   TC_Perl_In#test_perl_test_cases:
   ArgumentError: Could not find <rake_test_loader.xml> in </usr/lib64/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake>
   /var/tmp/portage/dev-ruby/xml-simple-1.1.5/work/ruby21/xml-simple-1.1.5/lib/xmlsimple.rb:998:in find_xml_file' /var/tmp/portage/dev-ruby/xml-simple-1.1.5/work/ruby21/xml-simple-1.1.5/lib/xmlsimple.rb:168:inxml_in'
   /var/tmp/portage/dev-ruby/xml-simple-1.1.5/work/ruby21/xml-simple-1.1.5/lib/xmlsimple.rb:203:in xml_in' /var/tmp/portage/dev-ruby/xml-simple-1.1.5/work/ruby21/xml-simple-1.1.5/test/tc_perl_in.rb:413:intest_perl_test_cases'
   /usr/lib64/ruby/2.1.0/minitest/unit.rb:1265:in run' /usr/lib64/ruby/2.1.0/test/unit/testcase.rb:17:inrun'
   /usr/lib64/ruby/2.1.0/minitest/unit.rb:940:in block in _run_suite' /usr/lib64/ruby/2.1.0/minitest/unit.rb:933:inmap'
   /usr/lib64/ruby/2.1.0/minitest/unit.rb:933:in _run_suite' /usr/lib64/ruby/2.1.0/test/unit.rb:663:inblock in _run_suites'
   /usr/lib64/ruby/2.1.0/test/unit.rb:661:in each' /usr/lib64/ruby/2.1.0/test/unit.rb:661:in_run_suites'
   /usr/lib64/ruby/2.1.0/minitest/unit.rb:884:in _run_anything' /usr/lib64/ruby/2.1.0/minitest/unit.rb:1092:inrun_tests'
   /usr/lib64/ruby/2.1.0/minitest/unit.rb:1079:in block in _run' /usr/lib64/ruby/2.1.0/minitest/unit.rb:1078:ineach'
   /usr/lib64/ruby/2.1.0/minitest/unit.rb:1078:in _run' /usr/lib64/ruby/2.1.0/minitest/unit.rb:1066:inrun'
   /usr/lib64/ruby/2.1.0/test/unit.rb:27:in run' /usr/lib64/ruby/2.1.0/test/unit.rb:780:inrun'
   /usr/lib64/ruby/2.1.0/test/unit.rb:372:in block (2 levels) in autorun' /usr/lib64/ruby/2.1.0/test/unit.rb:33:inrun_once'
   /usr/lib64/ruby/2.1.0/test/unit.rb:371:in `block in autorun'

Finished tests in 10.304425s, 3.6877 tests/s, 31.6369 assertions/s.
38 tests, 326 assertions, 0 failures, 1 errors, 0 skips

ruby -v: ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-linux]
rake aborted!
Command failed with status (1): [ruby -I"lib:./lib" -I"/usr/lib64/ruby/gems/2.1.0/gems/rake-10.4.2/lib" "/usr/lib64/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/rake_test_loader.rb" "test/tc_attr_prefix.rb" "test/tc_curtis.rb" "test/tc_file_in.rb" "test/tc_markup.rb" "test/tc_nick.rb" "test/tc_out.rb" "test/tc_perl_in.rb" "test/tc_perl_mem_copy.rb" "test/tc_perl_mem_share.rb" "test/tc_perl_out.rb" "test/tc_perl_storable.rb" "test/tc_string_in.rb" "test/tc_symbol.rb" ]

Add tags for releases on github

Please add tags on github.

Current gems don't include tests. I'd like to run them and download the tagged version instead of the gem file.

xml_in is vulnerable to file inclusion if misused in request

The method xml_in can be used to read system files in XML format instead of the expected user input.

Proof of Concept

Lets suppose we have in Sinatra the following code:

require 'sinatra'
require 'xmlsimple'
require 'json'

post '/echo_xml' do
  parser = XmlSimple.new
  data = parser.xml_in(request.body.read, {})
  JSON.pretty_generate(data)
end

If we do:

➜  curl http://localhost:4567/echo_xml -X POST --data "<a>hello</a>"
"hello"%

And lets suppose that we also have a file in the same directory named "secrets.xml", with the following content:

<password>S3CR3T</password

If we do:

➜  curl http://localhost:4567/echo_xml -X POST --data "secrets.xml"
"S3CR3T"%

It is very easy to forget to validate the input, causing potential vulnerabilities when parsing input received by the clients.

nil instead of {}

Hi,

When I do this:

XmlSimple.xml_in('<fiche><generation/></fiche>', 'ForceArray' => false)

I get this in return:

{"generation"=>{}}

I want to get:

{"generation"=>nil}

How can I do it?

Thanks for your help.

Is there any documentation about available options?

Hi! I'm trying to use this library, that seems very cool, but I'm having problems when doing xml_in when the xml has an empty tag. I got an empty hash {} as the value.

Is there any doc for being able understand how to interpret that as an empty string?

Thanks!

suppressempty does not suppress empty nodes if parent has attributes

If we are iterating over the children of an element, the suppressempty option should do so if the child has no attributes.

       element.each_element { |child|
         value = collapse(child)
-         if empty(value) && (element.attributes.empty? || @options['noattr'])
+         if empty(value) && (child.attributes.empty? || @options['noattr'])
           next if @options.has_key?('suppressempty') && @options['suppressempty'] == true
         end
         result = merge(result, child.name, value)
       }

please release

I need the license from the gemspec to show up on rubygems for our organization license audit

@maik

Content must be a string?

I noticed that unless content is a string, it's treated as an attribute. Is this a bug or a feature?

[13] pry(main)> XmlSimple.xml_out({y: {'content' => 'one'} })
=> "<opt>\n  <y>one</y>\n</opt>\n"
[14] pry(main)> XmlSimple.xml_out({y: {content: 'one'} })
=> "<opt>\n  <y content=\"one\" />\n</opt>\n"

When text contains newlines mixed content is false

I think I found a bug in https://github.com/maik/xml-simple/blob/master/lib/xmlsimple.rb#L725

When the xml is:

<h1 id="product-foo">
<span class="highlight">Product</span> Foo</h1>

The result of element.texts.join('') was "\n Foo", which causes /^\s*$/ to return false. A better regexp could be /\A\s*\z/ which returns true as expected. Alternatively something like this would also work !element.texts.join('').strip.empty?.

The specific method could be simplified down to:

def has_mixed_content?(element)
  element.has_text? &&
  element.has_elements? &&
  !element.texts.join('').strip.empty?
end

Interestingly enough I found this bug when testing the same code under JRuby and MRI. Under JRuby things (probably incorrectly) worked, and #has_mixed_content? returned true, while under MRI it returned false. I don't yet know the root cause of this difference, but I can confirm the change above fixes my issues.