willemdj / erlsom Goto Github PK

View Code? Open in Web Editor NEW

263.0 16.0 103.0 778 KB

XML parser for Erlang

License: GNU Lesser General Public License v3.0

Makefile 0.10% Erlang 99.75% Shell 0.15%

sax-parser erlang databinder xml xml-schema xsd

erlsom's People

Stargazers

Watchers

Forkers

jcomellas abanca noblesamurai sbalea devinus romanshestakov onlyafly srstrong chef unisontech siftlogic arcusfelis ztmr thomasbhatia tojans johanvikman mykook chinnurtb i11 eriksoe danielwhite namjae maciaslopez travelping jaydoane tigertext upstartmobile zacharyfox cmullaparthi taybrownob196 priyankabiban91 wangxin39 paulperegud alexdamir schlagert frobese ryderside processone kivra serge2 shuieryin mdabbagh88 djthread srenatus chenyu468 rbalint lemenkov platinumthinker spc16670 datafordevelopment mdemars amplify-social campanja-forks radardisc filmor kape1395 jomei ypcat francescorestelli matlc yuanyl kagesys wdyangkun jonathanhockman hampusbergstrom ates murtza varnerac cogini evgk davidalphafox neilberget ramondelemos ad-pro pherk baribeault buddasward oleks electronicru rogalskib kianmeng kenny-evitt terkiterje xiaoqianghan nordnet satyasyahputra neslinesli93 flambard arjitshandilya kuffel mikpe markomin 2600hz

erlsom's Issues

simple_form/2 don't keep attributes order according to the document's

When using simple_form/2 the order of the attributes are reversed in regarding the document's order.

Evidences

<?xml version="1.0" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
  "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg width="300px" height="100px" version="1.1"
     xmlns="http://www.w3.org/2000/svg">
  <desc>Example InitialCoords - SVG's initial coordinate system</desc>

  <g fill="none" stroke="black" stroke-width="3" >
    <line x1="0" y1="1.5" x2="300" y2="1.5" />
    <line x1="1.5" y1="0" x2="1.5" y2="100" />
  </g>
  <g fill="red" stroke="none" >
    <rect x="0" y="0" width="3" height="3" />
    <rect x="297" y="0" width="3" height="3" />
    <rect x="0" y="97" width="3" height="3" />
  </g>
  <g font-size="14" font-family="Verdana" >
    <text x="10" y="20">(0,0)</text>
    <text x="240" y="20">(300,0)</text>
    <text x="10" y="90">(0,100)</text>
  </g>
</svg>

{:ok,
 {'svg', [{'version', '1.1'}, {'height', '100px'}, {'width', '300px'}],
  [
    {'desc', [], ['Example InitialCoords - SVG\'s initial coordinate system']},
    {'g', [{'stroke-width', '3'}, {'stroke', 'black'}, {'fill', 'none'}],
     [
       {'line', [{'y2', '1.5'}, {'x2', '300'}, {'y1', '1.5'}, {'x1', '0'}], []},
       {'line', [{'y2', '100'}, {'x2', '1.5'}, {'y1', '0'}, {'x1', '1.5'}], []}
     ]},
    {'g', [{'stroke', 'none'}, {'fill', 'red'}],
     [
       {'rect', [{'height', '3'}, {'width', '3'}, {'y', '0'}, {'x', '0'}], []},
       {'rect', [{'height', '3'}, {'width', '3'}, {'y', '0'}, {'x', '297'}], []},
       {'rect', [{'height', '3'}, {'width', '3'}, {'y', '97'}, {'x', '0'}], []}
     ]},
    {'g', [{'font-family', 'Verdana'}, {'font-size', '14'}],
     [
       {'text', [{'y', '20'}, {'x', '10'}], ['(0,0)']},
       {'text', [{'y', '20'}, {'x', '240'}], ['(300,0)']},
       {'text', [{'y', '90'}, {'x', '10'}], ['(0,100)']}
     ]}
  ]}, '\n'}

Publish to hex package manager

Hi, Hex is a package manager for Elixir and Erlang and provides straightforward dependency and installation support.

I'm not an expert, but I believe to release to hex you need only add a file "package.exs" with the following content

defmodule Erlsom.Mixfile do
  use Mix.Project

  @version File.read!("vsn.mk")|> String.split("=") |> List.last |> String.strip

  def project do
    [app: :erlsom,
     version: @version,
     description: "XML parser for Erlang",
     package: package]
  end

  defp package do
    [files: ~w(include src rebar.config README.md COPYING COPYING.LESSER vsn.mk),
     contributors: ["Willem de Jong"],
     licenses: ["LGPL 3.0", "GPL 3.0"],
     links: %{"GitHub" => "https://github.com/willemdj/erlsom"}]
  end
end

Then follow the steps: https://hex.pm/docs/publish

Note, I believe you will need elixir installed to follow the publish steps? I would be happy to assist with this stage (but only the owner of a package on hex can then publish new versions, hence why I'm trying to avoid forking and publishing myself)

Thanks!

crash on encodingStyle attribute of a soap message

Hello.

I've got the next error using detergent:

{error,[{exception,{error,"Unexpected attribute (from target NS): encodingStyle"}},
        {stack,[undefined]},
        {received,{startElement,"http://schemas.xmlsoap.org/soap/envelope/",
                                "Envelope","SOAP-ENV",
                                [{attribute,"encodingStyle","SOAP-ENV",
                                            "http://schemas.xmlsoap.org/soap/envelope/",
                                            "http://schemas.xmlsoap.org/soap/encoding/"}]}}]}

Code to reproduce it (erl is run from detergent root directory):

{ok, Model} = erlsom:compile_xsd_file("priv/soap.xsd",
     [{prefix, "soap"},
      {include_files, [{"http://schemas.xmlsoap.org/wsdl/", "wsdl", "priv/wsdl.xsd"}]}]),
WsdlModel2 = erlsom:add_xsd_model(Model),
{ok, Env} = erlsom:compile_xsd_file("priv/envelope.xsd", [{prefix, "soap"}]),
SoapModel = erlsom:add_model(Env, Model),
{ok, XSD} = erlsom:compile_xsd_file("mywsdl.xsd"),
SoapModel2 = erlsom:add_model(XSD, SoapModel),
erlsom:scan_file("resp.xml", SoapModel2).

mywsdl.xsd:

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" targetNamespace="https://api-iz.merlion.ru/v2/mlservice.php">
<xsd:element name="helloWorld">
  <xsd:complexType>
    <xsd:sequence>
      <xsd:element name="name" type="xsd:string"/>
    </xsd:sequence>
  </xsd:complexType>
</xsd:element>
<xsd:element name="helloWorldResponse">
  <xsd:complexType>
    <xsd:sequence>
      <xsd:element name="return" type="xsd:string"/>
    </xsd:sequence>
  </xsd:complexType>
</xsd:element>
</xsd:schema>

resp.xml:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="https://api-iz.merlion.ru/v2/mlservice.php" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
  <SOAP-ENV:Body>
    <ns1:helloWorldResponse>
      <return xsi:type="xsd:string">Hello, ddd kjkjk</return>
    </ns1:helloWorldResponse>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

If i comment out 1297-1298 lines of src/erlsom_parse.erl code, then everything is fine. encodingStyle is a standard attribute, so i don't understand why erlsom complain about it.

Missing element before end-tag

I have an problem with erlsom:scan(BinXml, Model):

** exception error: no match of right hand side value {error,[{exception,{error,"Missing element before end-tag: collation"}},
{stack,[collation,'collations/SEQ1',collations,'ldml/SEQ1',
ldml]},
{received,{endElement,[],"collation",[]}}]}

minOccurs, maxOccurs have wrong values in a model.
minOccurs, maxOccurs are "0" and "undefined" in xsd, but they are "1" and "1" in a model.

XML:
https://gist.github.com/1171290

XSD:
https://gist.github.com/1171271

Model:
https://gist.github.com/1171281

Invalid xmlns used when writing XML

Hi,

I have a scenario where the XML generated from a given XSD is invalid, due to the wrong xmlns used.

I have replicated this scenario in a small github project with all the relevant XSD and XML files included:
https://github.com/apauley/erlsom-schema-debugging

If I manually change the xmlns, the resulting XML file becomes valid.
Compare the files erlsom-request-original.xml with erlsom-request-changed.xml.

I have validated the XML requests with an external library (lxml in Python):

$ ./validate-xsd.py 
Sample validation passed

Failed validation: erlsom-request-original.xml
Passed validation: erlsom-request-changed.xml
Passed validation: soapui-request.xml

Difference between the passed and failed XML is just the xmlns:

$ diff -u erlsom-request-original.xml erlsom-request-changed.xml
--- erlsom-request-original.xml 2012-08-19 10:38:29.000000000 +0200
+++ erlsom-request-changed.xml  2012-08-19 10:51:24.000000000 +0200
@@ -1,17 +1,17 @@
 <ns0:InsuranceGetQuoteRequest xmlns:ns0="http://tempuri.org/">
   <ns0:message>
-    <P:Header xmlns:P="http://Intellimin.co.nz/OFS/integration/v1">
+    <P:Header xmlns:P="http://schemas.datacontract.org/2004/07/InsuranceInterface">
       <P:SupplierID>X123</P:SupplierID>
       <P:MSISDN>0027825551234</P:MSISDN>
       <P:MNO>cellc</P:MNO>
       <P:DateTimeStamp>2012-08-14T16:52:17+02:00</P:DateTimeStamp>
       <P:SessionKey>1344-938844-722163</P:SessionKey>
     </P:Header>
-    <P:Detail xmlns:P="http://Intellimin.co.nz/OFS/integration/v1">
+    <P:Detail xmlns:P="http://schemas.datacontract.org/2004/07/InsuranceInterface">
       <P:ClientAccountId>0027825551234</P:ClientAccountId>
       <P:ProductType>PrePaidJourney</P:ProductType>
     </P:Detail>
-    <P:Trailer xmlns:P="http://Intellimin.co.nz/OFS/integration/v1">
+    <P:Trailer xmlns:P="http://schemas.datacontract.org/2004/07/InsuranceInterface">
       <P:StatusCode/>
       <P:StatusDescription/>
       <P:ProcessedTimeIn/>

To see the specific XML being generated:

2> erlsomtest:run().
XMLRequest generated by erlsom (unformatted):
<ns0:InsuranceGetQuoteRequest xmlns:ns0="http://tempuri.org/"><ns0:message><P:Header xmlns:P="http://Intellimin.co.nz/OFS/integration/v1"><P:SupplierID>X123</P:SupplierID><P:MSISDN>0027825551234</P:MSISDN><P:MNO>cellc</P:MNO><P:DateTimeStamp>2012-08-14T16:52:17+02:00</P:DateTimeStamp><P:SessionKey>1344-938844-722163</P:SessionKey></P:Header><P:Detail xmlns:P="http://Intellimin.co.nz/OFS/integration/v1"><P:ClientAccountId>0027825551234</P:ClientAccountId><P:ProductType>PrePaidJourney</P:ProductType></P:Detail><P:Trailer xmlns:P="http://Intellimin.co.nz/OFS/integration/v1"><P:StatusCode></P:StatusCode><P:StatusDescription></P:StatusDescription><P:ProcessedTimeIn></P:ProcessedTimeIn><P:ProcessedTimeOut></P:ProcessedTimeOut></P:Trailer></ns0:message></ns0:InsuranceGetQuoteRequest>
ok

Build multi xsd file with perfix

Dear you,
I try to build a hrl file with many xsd files.
I include xsd files in only one xsd file and run function "write_xsd_hrl_file/3" but i cannot not custom prefix for each xsd.
For example
In my main xsd i put:
<xs:import namespace="urn:ietf:params:xml:ns:common-policy" schemaLocation="lib/vMMTEL-1.0/priv/common-policy.xsd"/>
How can i edit this xsd with prefix "cp". ( When i build , it's become the prefix "P".

strange use of http_uri:parse/1 in erlsom_lib:get_url/1

Any idea why there is a call to http_uri:parse/1 in erlsom_lib:get_url/1? I can't find anything similar in yaws_soap_lib.

http_uri is deprecated in OTP 23 so it would be good to get rid of it.

1.5.1 to hex.pm

Hi!

Push 1.5.1 version to hex.pm please

Thank you!

regression bug on "any" elements

As far as I can tell ab14ec6 introduced a bug when handling "any" elements.
Using a ab14ec6_bug.xsd schema of (simplified from some SOAP and WSDL):

<?xml version="1.0" encoding="UTF-8"?>
<schema targetNamespace="urn:example:tns"
    xmlns="http://www.w3.org/2001/XMLSchema"
    xmlns:tns="urn:example:tns"
    elementFormDefault="qualified" attributeFormDefault="unqualified">

  <element name="Body" type="tns:Body" />
  <complexType name="Body" >
    <sequence>
      <any namespace="##any" processContents="lax" minOccurs="0" maxOccurs="unbounded" />
    </sequence>
    <anyAttribute namespace="##other" processContents="lax" />
  </complexType>

  <complexType name="RequestBaseType" />

  <element name="SignRequest" type="tns:RequestBaseType"></element>

  <element name="ArchiveSubmissionRequest">
    <complexType>
      <complexContent>
        <extension base="tns:RequestBaseType" />
      </complexContent>
    </complexType>
  </element>

</schema>

I noticed:

% erl -pa ebin

{ok, Model} = erlsom:compile_xsd_file("ab14ec6_bug.xsd").
Request = {'ArchiveSubmissionRequest', undefined}.
Body = {'Body', undefined, [Request]}.
erlsom:write(Body, Model).

% expected: (works up to e2ab569)
% {ok,"<Body xmlns=\"urn:example:tns\"><ArchiveSubmissionRequest/></Body>"}

% actual: (broken since ab14ec6)
% {ok,"<Body xmlns=\"urn:example:tns\"><SignRequest xsi:type=\"ArchiveSubmissionRequest\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"/></Body>"}

I'll try to debug this further.

For quick reference, the model here is:

{model,[{type,'_document',sequence,
                  [{el,[{alt,'Body','Body',[],1,1,true,undefined},
                        {alt,'SignRequest','RequestBaseType',[],1,1,true,undefined},
                        {alt,'ArchiveSubmissionRequest','ArchiveSubmissionRequest',
                             [],1,1,true,undefined}],
                       1,1,undefined,3}],
                  [],undefined,undefined,1,1,1,false,undefined},
            {type,'ArchiveSubmissionRequest',sequence,[],[],undefined,
                  undefined,1,1,1,undefined,'ArchiveSubmissionRequest'},
            {type,'RequestBaseType',sequence,[],[],undefined,undefined,
                  1,1,1,undefined,undefined},
            {type,'Body',sequence,
                  [{el,[{alt,'#any',any,[],1,1,true,
                             {anyInfo,"lax","##any","urn:example:tns"}}],
                       0,unbound,undefined,3}],
                  [],
                  {anyAttr,"lax","##other","urn:example:tns"},
                  undefined,2,1,1,undefined,undefined}],
           [{ns,"http://www.w3.org/2001/XMLSchema","xsd"},
            {ns,"urn:example:tns",undefined}],
           "urn:example:tns",
           [{'ArchiveSubmissionRequest','RequestBaseType'}],
           true,skip}

Earlier it used to be:

{model,[{type,'_document',sequence,
                  [{el,[{alt,'Body','Body',[],1,1,true,undefined},
                        {alt,'SignRequest','RequestBaseType',[],1,1,true,undefined},
                        {alt,'ArchiveSubmissionRequest','ArchiveSubmissionRequest',
                             [],1,1,true,undefined}],
                       1,1,undefined,3}],
                  [],undefined,undefined,1,1,1,false,undefined},
            {type,'ArchiveSubmissionRequest',sequence,[],[],undefined,
                  undefined,1,1,1,undefined,'ArchiveSubmissionRequest'},
            {type,'RequestBaseType',sequence,[],[],undefined,undefined,
                  1,1,1,undefined,undefined},
            {type,'Body',sequence,
                  [{el,[{alt,'#any',any,[],1,1,true,
                             {anyInfo,"lax","##any","urn:example:tns"}},
                        {alt,'ArchiveSubmissionRequest','ArchiveSubmissionRequest',
                             [],1,1,true,undefined},
                        {alt,'SignRequest','RequestBaseType',[],1,1,true,undefined},
                        {alt,'Body','Body',[],1,1,true,undefined}],
                       0,unbound,undefined,3}],
                  [],
                  {anyAttr,"lax","##other","urn:example:tns"},
                  undefined,2,1,1,undefined,undefined}],
           [{ns,"http://www.w3.org/2001/XMLSchema","xsd"},
            {ns,"urn:example:tns",undefined}],
           "urn:example:tns",
           [{'ArchiveSubmissionRequest','RequestBaseType'}],
           true,skip}

scan incorrectly fails with error "Unexpected attribute: length"

test.xsd (Link)
test.xml (Link)

{ok, Model} = erlsom:compile_xsd_file("test.xsd"),
erlsom:scan_file("test.xml", Model).

                    {error,[{exception,{error,"Unexpected attribute: length"}},
                            {stack,[descriptionsType,
                                    'partnerContent/seriesItems/series',
                                    'partnerContent/seriesItems',partnerContent]},
                            {received,{startElement,[],"description",[],
                                                    [{attribute,"length",[],[],"60"}]}}]}

W3C schema causes 'Unexpected event, expected end-tag'

Hello,

I'm experiencing an issue when compiling XSD that includes some of the W3C schema.

iex(4)> :erlsom.compile_xsd_file <FILE PATH>
{:error,
 [exception: {:error,
   {'2 - Unexpected event, expected end-tag'}},
  stack: [:schemaType],
  received: {:startElement, 'http://www.w3.org/2001/XMLSchema', 
   'notation', 'xs',
   [{:attribute, 'system', [], [],
     'http://www.w3.org/2000/08/XMLSchema.xsd'},
    {:attribute, 'public', [], [], 'structures'},
    {:attribute, 'name', [], [], 'XMLSchemaStructures'}]}]}

These appear to be the lines that break compilation:

<xs:import namespace="http://www.w3.org/2000/09/xmldsig#" schemaLocation="http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/xmldsig-core-schema.xsd"/>
<xs:import namespace="http://www.w3.org/1999/XSL/Transform" schemaLocation="http://www.w3.org/2007/schema-for-xslt20.xsd"/>

I also get the same error when I download and attempt compilation on those schemas directly.

This may be related to #45 and this exchange from 2009. If that is the case, has a workaround found?

Thanks! 😄

Recursive entities can cause erlsom to parse, effectively, forever

Testcase:

erlsom:simple_form(<<"\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a]>\x0d\x0a&x;">>).

This is problematic if parsing XML from the outside world.

Is there a known fix for this in erlsom? I did not see one.

Here is a link to the general description of the attack:

http://projects.webappsec.org/w/page/13247002/XML%20Entity%20Expansion

simple_form does not handle multiple root tags correctly

There seems to be a bug in erlsom when you have an errornous xml with multiple root tags.

erlsom:simple_form("<donald>duck</donald><uncle>scrooge</uncle>").
{ok,{"donald",[],[{"uncle",[],["scrooge"]},"duck"]},[]}

I would have expected

{ok,{"donald",[],["duck"]},"<uncle>scrooge</uncle>"}

'Type not found in hierarchy' when no type is defined and XSI type is given

Given the following XSD (extracted from XAdES)

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
	<xs:element name="SignedSignatureProperties" type="SignedSignaturePropertiesType"/>
	<xs:complexType name="SignedSignaturePropertiesType">
		<xs:sequence>
			<xs:element name="SigningTime" type="xs:dateTime" minOccurs="0"/>
			<xs:element name="SignaturePolicyIdentifier" type="SignaturePolicyIdentifierType" minOccurs="0"/>
		</xs:sequence>
		<xs:attribute name="Id" type="xs:ID" use="optional"/>
	</xs:complexType>
	<xs:complexType name="SignaturePolicyIdentifierType">
		<xs:choice>
			<xs:element name="SignaturePolicyId" type="SignaturePolicyIdType"/>
			<xs:element name="SignaturePolicyImplied"/>
		</xs:choice>
	</xs:complexType>
	<xs:complexType name="SignaturePolicyIdType">
		<xs:sequence>
		</xs:sequence>
	</xs:complexType>
	<xs:complexType name="SPUserNoticeType">
		<xs:sequence>
			<xs:element name="NoticeRef" type="NoticeReferenceType" minOccurs="0"/>
			<xs:element name="ExplicitText" type="xs:string" minOccurs="0"/>
		</xs:sequence>
	</xs:complexType>
	<xs:complexType name="NoticeReferenceType">
		<xs:sequence>
			<xs:element name="Organization" type="xs:string"/>
			<xs:element name="NoticeNumbers" type="IntegerListType"/>
		</xs:sequence>
	</xs:complexType>
	<xs:complexType name="IntegerListType">
		<xs:sequence>
			<xs:element name="int" type="xs:integer" minOccurs="0" maxOccurs="unbounded"/>
		</xs:sequence>
	</xs:complexType>
</xs:schema>

And the following response

<xades:SignedSignatureProperties xmlns:xades="http://uri.etsi.org/01903/v1.3.2#">
	<xades:SigningTime>2001-10-26T21:32:52+02:00</xades:SigningTime>
	<xades:SignaturePolicyIdentifier>
		<xades:SignaturePolicyImplied xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xades:SPUserNoticeType">
			<xades:ExplicitText>Advanced electronic signature</xades:ExplicitText>
		</xades:SignaturePolicyImplied>
	</xades:SignaturePolicyIdentifier>
</xades:SignedSignatureProperties>

You will get an error when scanning the response:

1> {ok, Model} = erlsom:compile_xsd_file("priv/example.xsd").
2> erlsom:scan_file("priv/response.xml", Model)
{error,[{exception,{error,"Type not found in hierarchy: SignaturePolicyIdentifierType-SignaturePolicyImplied"}},
        {stack,['SignaturePolicyIdentifierType',
                'SignedSignaturePropertiesType']},
        {received,{startElement,"http://uri.etsi.org/01903/v1.3.2#",
                                "SignaturePolicyImplied","xades",
                                [{attribute,"type","xsi",
                                            "http://www.w3.org/2001/XMLSchema-instance",
                                            "xades:SPUserNoticeType"}]}}]}

I was able to narrow the issue down to translateType not adding the type to the history, because it was not defined and findDerivedType therefore being unable to find the type as ancestor in the TypeHierarchy.

I'm not so familiar with XML-Schemas in general, but is this check necessary?

Parsing large PCDATA

Hello,

Parsing an xml file with a large PCDATA content (~100 MB, base64 encoded string) with erlsom:scan, seems to need about 10 GB of memory (100 times as much!); no matter whether I pass it a string (char list) or a binary.
Is that an 'expected behaviour', or am I using it in wrong way?

Thanks, David.

Hex docs are broken (not setup?)

The HexDocs for this package show "PAGE NOT FOUND":

https://hexdocs.pm/erlsom/

That's the link for "Online documentation" on the Hex page for this project's package:

erlsom | Hex

If Hex docs were never setup, I'd be willing to submit a PR to do that.

Thanks for the library!

XML signature

Hello,

I have a question about create an signature in xml using this tool, how can I to do it?

Thanks,

Pacheco.

Build is broken on macOS: escript: exception error: undefined function rebar:main/1

--->  Configuring erlsom
--->  Building erlsom
Executing:  cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_erlang_erlsom/erlsom/work/erlsom-e00d2c7426fe7a8c49f8959da38a11d21b8b2168" && /usr/bin/make -j6 -w all CC="/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_erlang_erlsom/erlsom/work/compwrap/cc/opt/local/bin/gcc-mp-12" CXX="/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_erlang_erlsom/erlsom/work/compwrap/cxx/opt/local/bin/g++-mp-12" OBJC="/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_erlang_erlsom/erlsom/work/compwrap/objc/opt/local/bin/gcc-mp-12" OBJCXX="/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_erlang_erlsom/erlsom/work/compwrap/objcxx/opt/local/bin/g++-mp-12" FC="/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_erlang_erlsom/erlsom/work/compwrap/fc/opt/local/bin/gfortran-mp-12" F77="/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_erlang_erlsom/erlsom/work/compwrap/f77/opt/local/bin/gfortran-mp-12" F90="/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_erlang_erlsom/erlsom/work/compwrap/f90/opt/local/bin/gfortran-mp-12" INSTALL="/usr/bin/install -c" 
make: Entering directory `/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_erlang_erlsom/erlsom/work/erlsom-e00d2c7426fe7a8c49f8959da38a11d21b8b2168'
escript: exception error: undefined function rebar:main/1
  in function  escript:run/2 (escript.erl, line 750)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_em/1 
  in call from init:do_boot/3 
make: *** [compile] Error 127

Undefined function erlsom

Hi,

I'm working on ubuntu.
I clone the repo, and use the command "rebar compile" and it showed me this:
==> erlsom (compile)
After that, I moved all the .beam files to 'ebin' directory.
But when I try to run the continuation_example i get this error:
** exception error: undefined function erlsom:parse_sax/4
in function continuation_example:run/0 (continuation_example.erl, line 41)

The only thing I did different, is that these files are not in 'ROOT/lib/erlsom-1.2.1/src',
but when I checked it I saw that I don't have an erlsom directory in this ROOT.
Do I need to install something In order to have erlsome-1.2.1 directory?
(When I tried to clone the repo into erlang/lib I got 'Permission denied' message).

I'm using ubuntu 20.04 and erlamg OTP 22.

Thanks!

error: 1 - Unexpected event, expected end-tag

Hi.

Got the next error:

{error,{decoding,[{exception,{error,{"1 - Unexpected event, expected end-tag"}}},
                  {stack,['soap:detail','soap:Fault','soap:Body',
                          'soap:Envelope']},
                  {received,{characters,"Invalid login/pass"}}]}}

while processing the next soap xml response:

<?xml version="1.0" encoding="UTF-8"?>
  <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:ns2="urn://pmtv.digisoft.com/systemstatus/schema/v0.1" xmlns:ns3="urn://pmtv.digisoft.com/providerstatus/schema/v0.1" xmlns:ns5="http://schemas.microsoft.com/Message" xmlns:ns4="urn://pmtv.digisoft.com/uploadUsageRecords/wsdl/v0.1" xmlns:ns7="ws.kitd.vx2.ws.EntityNotification/EntityNotificationServiceSoap" xmlns:ns6="ws.kitd.vx2.ws.EntityNotification" xmlns:ns8="ws.kitd.vx2.ws.EntityNotification/EntityNotificationServiceSoap12" xmlns:ns11="http://schemas.microsoft.com/2003/10/Serialization/" xmlns:ns12="http://schemas.datacontract.org/2004/07/GMCS.PMT.IntegrationInternal" xmlns:ns15="http://schemas.microsoft.com/2003/10/Serialization/" xmlns:ns16="http://schemas.datacontract.org/2004/07/GMCS.PMT.IntegrationInternal" xmlns:ns9="http://GMCS/PMT/Integration" xmlns:lbapi="urn:api3">
    <SOAP-ENV:Body>
      <SOAP-ENV:Fault>
        <faultcode>SOAP-ENV:Server</faultcode>
        <faultstring>error_request</faultstring>
        <detail>Invalid login/pass</detail>
      </SOAP-ENV:Fault>
    </SOAP-ENV:Body>
  </SOAP-ENV:Envelope>

If i add the next line:

error_logger:info_msg("~p~n~p~n", [Event, ElementRecord]),

near 444 line of erlsom_parse.erl (before a throw call), i get the next output:

{characters,"Invalid login/pass"}
{'soap:detail',[],undefined}

How can i eliminate this error?

Thanks.

unsupported schema types

hi,

there is an issue with types which are defined in xml schema but not supported natively in erlang.

e.g.
wsdl definition could have the following:
<xsd:element name="betId" nillable="false" type="xsd:long"/>

the WSDL model will bind name "betId" with type "long", but "long" in erlang is represented with "integer", so print_value function in ersome_write will throw
throw({error, "Type of value (integer) does not match model"})

I think ersom xml model builder should transform xsd:long -> xsd:int , probably do the same with double to float

Regards, Roman

Problem using detergent

Hi,

I'm using detergent as a soap client and have this code:
SoapRes = detergent:call(Wsdl, "GenerateChallenge", [],
[#'p:GenerateChallengeRequest'{provider = Client},
#'p:GenerateChallengeRequest'{transactionID = "1"},
#'p:GenerateChallengeRequest'{policy = "Test"}]),

it gives me this erlsom error which I can't understand:
{error,{error,function_clause,
[{erlsom_write,processAnyAttributes,
[2,[],
[{ns,"http://schemas.xmlsoap.org/soap/envelope/",
"soap"},
{ns,"http://www.w3.org/2001/XMLSchema","xsd"},
{ns,"urn:www.sll.se/wsdl/soap/osif","p"}],
{[{"soap",
"http://schemas.xmlsoap.org/soap/envelope/"}],
0}]},
{erlsom_write,processAlternativeValue,8},
{erlsom_write,processElementValues,7},
{erlsom_write,struct2xml,6},
{erlsom_write,processAlternativeValue,8},
{erlsom_write,processElementValues,7},
{erlsom_write,struct2xml,6},
{erlsom_write,processAlternativeValue,8}]}}

My record looks like this:
-record('p:GenerateChallengeRequest', {'provider', 'transactionID', 'policy'}).

My record used to look like this:
-record('p:GenerateChallengeRequest', {anyAttribs, 'provider', 'transactionID', 'policy'}).

as autogenerated from write_hlr() but erlsom kept complaining about nil values so I changed it.

/dang

Breaking change in erlsom.simple_form/1 between 1.5.0 and 1.5.1

There is a breaking change in the output of erlsom:simple_form that was introduced with this merge: #77

Is it possible to introduce an option to make this behavior configurable?

Before:

{:model,
 [
   {:type, :_document, :sequence,
    [
      {:el,
       [
         {:alt, :otherComplexType, :otherComplexType, [], 1, 1, true,
          :undefined}
       ], 1, 1, :undefined, 2}
    ], [], :undefined, :undefined, 1, 1, 1, false, :undefined},
   {:type, :"otherComplexType-someChoiceA", :sequence, [],
    [{:att, :type, 2, false, :char}, {:att, :time, 3, false, :char}],
    :undefined, :undefined, 3, 1, 1, :undefined, :undefined},
   {:type, :"otherComplexType-someChoiceB", :sequence, [],
    [{:att, :type, 2, false, :char}, {:att, :time, 3, false, :char}],
    :undefined, :undefined, 3, 1, 1, :undefined, :undefined},
   {:type, :otherComplexType, :sequence,
    [
      {:el,
       [
         {:alt, :someChoiceB, :"otherComplexType-someChoiceB", [], 1, 1, true,
          :undefined},
         {:alt, :someChoiceA, :"otherComplexType-someChoiceA", [], 1, 1, true,
          :undefined}
       ], 1, 1, :undefined, 2}
    ], [], :undefined, :undefined, 2, 1, 1, :undefined, :undefined},
   {:type, :someComplexType, :sequence, [],
    [{:att, :type, 2, false, :char}, {:att, :time, 3, false, :char}],
    :undefined, :undefined, 3, 1, 1, :undefined, :undefined}
 ],
 [
   {:ns, ~c"http://www.w3.org/2001/XMLSchema", ~c"xsd", :qualified},
   {:ns, ~c"urn:minimal", :undefined, :unqualified}
 ], ~c"urn:minimal", [], false, :skip}

After:

{:model,
 [
   {:type, :_document, :sequence,
    [
      {:el,
       [
         {:alt, :otherComplexType, :otherComplexType, [], 1, 1, true,
          :undefined}
       ], 1, 1, :undefined, 2}
    ], [], :undefined, :undefined, 1, 1, 1, false, :undefined},
   {:type, :otherComplexType_someChoiceA, :sequence,
    [
      {:el,
       [{:alt, :someChoiceA, :someComplexType, [], 1, 1, true, :undefined}], 1,
       1, :undefined, 2}
    ], [], :undefined, :undefined, 2, 1, 1, false, :undefined},
   {:type, :otherComplexType_someChoiceB, :sequence,
    [
      {:el,
       [{:alt, :someChoiceB, :someComplexType, [], 1, 1, true, :undefined}], 1,
       1, :undefined, 2}
    ], [], :undefined, :undefined, 2, 1, 1, false, :undefined},
   {:type, :otherComplexType, :sequence,
    [
      {:el,
       [
         {:alt, :someChoiceB, :otherComplexType_someChoiceB, [], 1, 1, false,
          :undefined},
         {:alt, :someChoiceA, :otherComplexType_someChoiceA, [], 1, 1, false,
          :undefined}
       ], 1, 1, :undefined, 2}
    ], [], :undefined, :undefined, 2, 1, 1, :undefined, :undefined},
   {:type, :someComplexType, :sequence, [],
    [{:att, :type, 2, false, :char}, {:att, :time, 3, false, :char}],
    :undefined, :undefined, 3, 1, 1, :undefined, :undefined}
 ],
 [
   {:ns, ~c"http://www.w3.org/2001/XMLSchema", ~c"xsd", :qualified},
   {:ns, ~c"urn:minimal", :undefined, :unqualified}
 ], ~c"urn:minimal", [], false, :skip}

erlsom_write cannot create XML document

Hi, I try to create SOAP client with this hrl file:

-record(faultdetail, {uri :: string(),
                      tag :: string(),
                      text :: string()}).

-record(faultcode, {uri :: string(),
                    code :: string(),
                    subcode :: #faultcode{} % only v. 1.2
                   }).

-record(faultreason, {text :: string(),
                      language :: string()}).

-record(soap_fault_1_1, {faultcode :: #faultcode{},
                         faultstring :: string(),
                         faultactor :: string(),
                         detail :: [#faultdetail{}]}).

-record(soap_fault_1_2, {code :: #faultcode{},
                         reason :: [#faultreason{}],
                         role :: string(),
                         detail :: [#faultdetail{}]}).

%% xsd:QName values are translated to #qname{} records.
-record(qname, {uri :: string(),
                localPart :: string(),
                prefix :: string(),
                mappedPrefix :: string()}).



-record('P0:Pair', {
	'Name' :: string(),
	'Value' :: string()}).

-type 'P0:Pair'() :: #'P0:Pair'{}.


-record('P0:RowData', {
	'RowData' :: ['P0:Pair'()]}).

-type 'P0:RowData'() :: #'P0:RowData'{}.


-record('P0:ColumnNames', {
	'Column' :: [string()]}).

-type 'P0:ColumnNames'() :: #'P0:ColumnNames'{}.


-record('P0:DataSet', {
	'Columns' :: 'P0:ColumnNames'(),
	'Row' :: ['P0:RowData'()] | undefined}).

-type 'P0:DataSet'() :: #'P0:DataSet'{}.


-record('P0:getBIDataResponse', {
	'Message' :: string(),
	'Data' :: 'P0:DataSet'() | undefined}).

-type 'P0:getBIDataResponse'() :: #'P0:getBIDataResponse'{}.


-record('P0:getBIData', {
	'MetricName' :: string(),
	'ClientID' :: string(),
	'Username' :: string(),
	'Password' :: string(),
	'FromDate' :: string() | undefined,
	'ToDate' :: string() | undefined,
	'Parameters' :: [string()] | undefined}).

-type 'P0:getBIData'() :: #'P0:getBIData'{}.
-define(INTERFACE, {interface,"BIData",'BIData','1.1',soap_client_inets,
                    soap_server_cowboy_1,undefined,'BIData_wsdl_client',[],
                    "https://ws.jobdiva.com/BIData/",
                    "http://schemas.xmlsoap.org/soap/envelope/","document",
                    undefined,"https://ws.jobdiva.com/axis2/services/BIData/",
                    "BIDataSOAP","BIDataSOAP","BIData",
                    [{op,"getBIData",getBIData,
                      "https://ws.jobdiva.com/BIData/getBIData",undefined,
                      request_response,'P0:getBIData','P0:getBIDataResponse',
                      undefined}],
                    {model,
                     [{type,'_document',sequence,
                       [{el,
                         [{alt,'P0:getBIData','P0:getBIData',[],1,1,true,
                           undefined},
                          {alt,'P0:getBIDataResponse','P0:getBIDataResponse',
                           [],1,1,true,undefined}],
                         1,1,undefined,2}],
                       [],undefined,undefined,1,1,1,false,undefined},
                      {type,'P0:Pair',sequence,
                       [{el,
                         [{alt,'Name',{'#PCDATA',char},[],1,1,true,undefined}],
                         1,1,undefined,2},
                        {el,
                         [{alt,'Value',
                           {'#PCDATA',char},
                           [],1,1,true,undefined}],
                         1,1,undefined,3}],
                       [],undefined,undefined,3,1,1,undefined,undefined},
                      {type,'P0:RowData',sequence,
                       [{el,
                         [{alt,'RowData','P0:Pair',[],1,1,true,undefined}],
                         1,unbound,undefined,2}],
                       [],undefined,undefined,2,1,1,undefined,undefined},
                      {type,'P0:ColumnNames',sequence,
                       [{el,
                         [{alt,'Column',
                           {'#PCDATA',char},
                           [],1,1,true,undefined}],
                         1,unbound,undefined,2}],
                       [],undefined,undefined,2,1,1,undefined,undefined},
                      {type,'P0:DataSet',sequence,
                       [{el,
                         [{alt,'Columns','P0:ColumnNames',[],1,1,true,
                           undefined}],
                         1,1,undefined,2},
                        {el,
                         [{alt,'Row','P0:RowData',[],1,1,true,undefined}],
                         0,unbound,undefined,3}],
                       [],undefined,undefined,3,1,1,undefined,undefined},
                      {type,'P0:getBIDataResponse',sequence,
                       [{el,
                         [{alt,'Message',
                           {'#PCDATA',char},
                           [],1,1,true,undefined}],
                         1,1,undefined,2},
                        {el,
                         [{alt,'Data','P0:DataSet',[],1,1,true,undefined}],
                         0,1,undefined,3}],
                       [],undefined,undefined,3,1,1,undefined,undefined},
                      {type,'P0:getBIData',sequence,
                       [{el,
                         [{alt,'MetricName',
                           {'#PCDATA',char},
                           [],1,1,true,undefined}],
                         1,1,undefined,2},
                        {el,
                         [{alt,'ClientID',
                           {'#PCDATA',char},
                           [],1,1,true,undefined}],
                         1,1,undefined,3},
                        {el,
                         [{alt,'Username',
                           {'#PCDATA',char},
                           [],1,1,true,undefined}],
                         1,1,undefined,4},
                        {el,
                         [{alt,'Password',
                           {'#PCDATA',char},
                           [],1,1,true,undefined}],
                         1,1,undefined,5},
                        {el,
                         [{alt,'FromDate',
                           {'#PCDATA',char},
                           [],1,1,true,undefined}],
                         0,1,undefined,6},
                        {el,
                         [{alt,'ToDate',
                           {'#PCDATA',char},
                           [],1,1,true,undefined}],
                         0,1,undefined,7},
                        {el,
                         [{alt,'Parameters',
                           {'#PCDATA',char},
                           [],1,1,true,undefined}],
                         0,unbound,undefined,8}],
                       [],undefined,undefined,8,1,1,undefined,undefined}],
                     [{ns,"https://ws.jobdiva.com/BIData/","P0",unqualified},
                      {ns,"http://www.w3.org/2001/XMLSchema","xsd",qualified}],
                     "https://ws.jobdiva.com/BIData/",[],false,skip},
                    1,undefined,
                    [{"https://ws.jobdiva.com/BIData/","P0"}]}).

I use bet365/soap that call erlsom_write.write function. That function throw error. I debbug code and I found that call of function findAlternative/3 with params:

< RecordType = <<"P0:getBIData">>

< Alternatives = [{alt,'P0:getBIData','P0:getBIData',[],1,1,true,undefined},
                  {alt,'P0:getBIDataResponse','P0:getBIDataResponse',[],1,1,
                       true,undefined}]

< Model = {model,[{type,'_document',sequence,
                        [{el,[{alt,'P0:getBIData','P0:getBIData',[],1,1,true,
                                   undefined},
                              {alt,'P0:getBIDataResponse',
                                   'P0:getBIDataResponse',[],1,1,true,
                                   undefined}],
                             1,1,undefined,2}],
                        [],undefined,undefined,1,1,1,false,undefined},
                  {type,'P0:Pair',sequence,
                        [{el,[{alt,'Name',
                                   {'#PCDATA',char},
                                   [],1,1,true,undefined}],
                             1,1,undefined,2},
                         {el,[{alt,'Value',
                                   {'#PCDATA',char},
                                   [],1,1,true,undefined}],
                             1,1,undefined,3}],
                        [],undefined,undefined,3,1,1,undefined,undefined},
                  {type,'P0:RowData',sequence,
                        [{el,[{alt,'RowData','P0:Pair',[],1,1,true,undefined}],
                             1,unbound,undefined,2}],
                        [],undefined,undefined,2,1,1,undefined,undefined},
                  {type,'P0:ColumnNames',sequence,
                        [{el,[{alt,'Column',
                                   {'#PCDATA',char},
                                   [],1,1,true,undefined}],
                             1,unbound,undefined,2}],
                        [],undefined,undefined,2,1,1,undefined,undefined},
                  {type,'P0:DataSet',sequence,
                        [{el,[{alt,'Columns','P0:ColumnNames',[],1,1,true,
                                   undefined}],
                             1,1,undefined,2},
                         {el,[{alt,'Row','P0:RowData',[],1,1,true,undefined}],
                             0,unbound,undefined,3}],
                        [],undefined,undefined,3,1,1,undefined,undefined},
                  {type,'P0:getBIDataResponse',sequence,
                        [{el,[{alt,'Message',
                                   {'#PCDATA',char},
                                   [],1,1,true,undefined}],
                             1,1,undefined,2},
                         {el,[{alt,'Data','P0:DataSet',[],1,1,true,undefined}],
                             0,1,undefined,3}],
                        [],undefined,undefined,3,1,1,undefined,undefined},
                  {type,'P0:getBIData',sequence,
                        [{el,[{alt,'MetricName',
                                   {'#PCDATA',char},
                                   [],1,1,true,undefined}],
                             1,1,undefined,2},
                         {el,[{alt,'ClientID',
                                   {'#PCDATA',char},
                                   [],1,1,true,undefined}],
                             1,1,undefined,3},
                         {el,[{alt,'Username',
                                   {'#PCDATA',char},
                                   [],1,1,true,undefined}],
                             1,1,undefined,4},
                         {el,[{alt,'Password',
                                   {'#PCDATA',char},
                                   [],1,1,true,undefined}],
                             1,1,undefined,5},
                         {el,[{alt,'FromDate',
                                   {'#PCDATA',char},
                                   [],1,1,true,undefined}],
                             0,1,undefined,6},
                         {el,[{alt,'ToDate',
                                   {'#PCDATA',char},
                                   [],1,1,true,undefined}],
                             0,1,undefined,7},
                         {el,[{alt,'Parameters',
                                   {'#PCDATA',char},
                                   [],1,1,true,undefined}],
                             0,unbound,undefined,8}],
                        [],undefined,undefined,8,1,1,undefined,undefined}],
                 [{ns,"http://ws.jobdiva.com/BIData/","P0",unqualified},
                  {ns,"http://www.w3.org/2001/XMLSchema","xsd",qualified}],
                 "http://ws.jobdiva.com/BIData/",[],false,skip}

exit with error with message: Struct doesn't match model: recordtype not expected: P0:getBIData

That provides to error on soap client and finally to error in my app.

Problem

It's look like erlsom_write:findAlternative/3 try to find parent elemnt of root element in XML.

Exception error: no match of right hand side value

Hi, I have the following error:

** exception error: no match of right hand side value 
                    {error,
                     {'EXIT',
                      {error,
                       [{exception,
                         {{badmatch,[]},
                          [{erlsom_parse,insertValue,2,
                            [{file,"src/erlsom_parse.erl"},{line,1171}]},
                           {erlsom_parse,stateMachine,2,
                            [{file,"src/erlsom_parse.erl"},{line,717}]},
                           {erlsom_parse,xml2StructCallback,2,
                            [{file,"src/erlsom_parse.erl"},{line,305}]},
                           {erlsom_sax_utf8,wrapCallback,2,
                            [{file,"src/erlsom_sax_utf8.erl"},{line,1408}]},
                           {erlsom_sax_utf8,parseContentLT,2,
                            [{file,"src/erlsom_sax_utf8.erl"},{line,907}]},
                           {erlsom_sax_utf8,parse,2,
                            [{file,"src/erlsom_sax_utf8.erl"},{line,196}]},
                           {erlsom,scan2,3,
                            [{file,"src/erlsom.erl"},{line,212}]},
                           {erlsom,scan_file,3,
                            [{file,"src/erlsom.erl"},{line,228}]}]}},
                        {stack,
                         [{'#text',char,"playannouncement"},
                          paramType,mscivrType]},
                        {received,{endElement,[],"param",[]}}]}}}

When I try open the following xml file:

<?xml version="1.0" encoding="UTF-8"?>
<mscivr xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="msc-ivr.xsd">
    <param name="mode">playannouncement</param>
</mscivr>

with xsd schema:

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">

    <xsd:complexType name="mscivrType">
        <xsd:sequence>
                <xsd:element name="param" type="paramType"/>
        </xsd:sequence>
    </xsd:complexType>

    <xsd:element name="mscivr" type="mscivrType"/>

    <xsd:complexType name="paramType" mixed="true">
        <xsd:attribute name="name" type="xsd:string" use="required"/>
    </xsd:complexType>
</xsd:schema>

If you replace the scheme on

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">

    <xsd:complexType name="mscivrType">
        <xsd:sequence>
                <xsd:element name="param" type="paramType"/>
        </xsd:sequence>
    </xsd:complexType>

    <xsd:element name="mscivr" type="mscivrType"/>

    <xsd:complexType name="paramType">
        <xsd:simpleContent>
            <xsd:extension base="xsd:string"/>
        </xsd:simpleContent>
        <xsd:attribute name="name" type="xsd:string" use="required"/>
    </xsd:complexType>
</xsd:schema>

we get the following result

{mscivrType,[{{"schemaLocation","http://www.w3.org/2001/XMLSchema-instance"},
                "msc-ivr.xsd"}],
              {paramType,[],"mode","playannouncement"}}

That result I expected in the first case.
I think that the problem in parsing mixed type.
I would like to hear your comments on that.
Thanks.

Restrictions in model when using compile_xsd_file

Is it possible to get the restrictions from the XSD file when using compile_xsd_file function?

Example XSD

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <xsd:complexType name="restrictions_test">
    <xsd:sequence>
      <xsd:element name="range" >
        <xsd:simpleType>
          <xsd:restriction base="xsd:integer" >
            <xsd:minInclusive value="0"/>
            <xsd:maxInclusive value="42"/>
          </xsd:restriction>
        </xsd:simpleType>
      </xsd:element>
    </xsd:sequence>
  </xsd:complexType>
</xsd:schema>

The model returned by compile_xsd_file does not contain the restriction:

{:model,
 [
   {:type, :_document, :sequence, [{:el, [], 1, 1, :undefined, 2}], [],
    :undefined, :undefined, 1, 1, 1, false, :undefined},
   {:type, :restrictions_test, :sequence,
    [
      {:el, [{:alt, :range, {:"#PCDATA", :char}, [], 1, 1, true, :undefined}],
       1, 1, :undefined, 2}
    ], [], :undefined, :undefined, 2, 1, 1, :undefined, :undefined}
 ], [{:ns, 'http://www.w3.org/2001/XMLSchema', 'xsd', :qualified}], :undefined,
 [], false, :skip}

Is there any option to allow getting the content of the restriction?

XML parsing fails for ihsmarkit blog rss feed

Parsing fails for ihsmarkit blog rss feed.

** exception throw: {error,"Malformed: Illegal character in prolog"}

Steps to reproduce the issue is as follows:

application:ensure_all_started(inets),
application:ensure_all_started(ssl),

Url = "https://ihsmarkit.com/BlogFeed.ashx?i=Technology",
Headers = [], ConnectTimeoutMsec = 2000, TimeoutMsec = 2000,
HttpOptions = [{timeout, TimeoutMsec}, {connect_timeout, ConnectTimeoutMsec}],
{ok, {{_, Code, _}, _Headers, Body}} = httpc:request(get, {Url, Headers}, HttpOptions, [{body_format, binary}]),

{ok, {XmlNode, _XmlAttribute, XmlValue}, _} = erlsom:simple_form(binary_to_list(Body)).

I am using the the following revision of erlsom in my project.

commit 1a9ea1a16ed2cf0466c11608367612d8b05ea6ae
Merge: df4526c b4ef336
Author: Willem de Jong <[email protected]>
Date:   Wed Feb 28 13:36:29 2018 +0100

Ambiguous documentation about CState and State in CFunction

In the sax mode documentation,

Option = {continuation_function, CState, CFunction} | ...

there is no documentation about the CState. Can it be anything?

Also, in the documentation of CFunction,

CFunction should be a function that takes 2 arguments: Tail and State.

Is the State the same thing as CState?

Add fetching included xsd files from remote locations

At the moment when including filed erlsom searches only in local directories. Most XSD files include other files from remote locations (usually through HTTP). I think this severly limits erlsom's functionality.

It appears that this is simply modifying getFile function in erlsom_lib.erl module

error: no match of right hand side value {error,[{exception,{error,"unknown tag: html"}

when use erlsom, following error came.
{'wsdl:tTypes','wsdl:tExtensibleDocumented'}]}
** exception error: no match of right hand side value {error,[{exception,{error,"unknown tag: html"}},
{stack,[undefined]},
{received,{startElement,[],"html",[],[]}}]}

Same WSDL file work with PHP nusoap lib.
Pls help to solve this

Remove inets application from application deps

Hi!

I noticed that "inets" is needlessly for erlsom, and I think it will be good idea to remove it from application deps.

Data binding: case insensitive parsing

We're trying to parse a few wsdl files in which they didn't camel-case the attribute names. So something like

erlsom:scan(WsdlFile, WsdlModel)

<wsdl:definitions name="MyService" targetnamespace="http://example.com/"  xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  ...
</wsdl:definitions>

will fail to parse with the xsd model we have 'Unexpected attribute: targetnamespace', because the correct definition is targetNamespace.

Would it be possible to parse these files case insensitive?

xsi:nil="true" & integer

Hello.

When i get a response from a soap server with empty elements (which marked by xsi:nil="true" attribute and nillable="true" attribute in a schema), erlsom crashes with the next error:

{error,{decoding,[{exception,{error,"Wrong Type: /integer"}},
                  {stack,[{'#PCDATA',integer,[]},
                          'p:ItemsAvailResult','p:ArrayOfItemsAvailResult',
                          'p:getItemsAvailResponse','soap:Body','soap:Envelope']},
                  {received,{endElement,[],"AvailableClient_RG",[]}}]}}

If i change an element type from int to string in the wsdl schema, everything works fine.

Make erlsom:compile_xsd_file work with XSDs generated with W3C DTD to XSD converter

Usually I create DTDs by hand and convert them to XSDs as these are easier to create without dedicated tool. (Though I am aware that XSDs are more accurate.)
I have created a DTD and validated it against an example XML files using xmllint.

When I convert the DTD into an XSD using the w3c DTD2Schema converter http://www.w3.org/2000/04/schema_hack/ perl script it fails. (I am assuming that the generated XSD is valid, if that assumption is not valid, please reject this issue.)
When however converting the DTD to XSD using MS Visual Studio erlsom:compile_xsd_file runs like charm.
Here the used DTD:

<!ELEMENT osm_traffic  (traffic_api)>
<!ATTLIST osm_traffic version            CDATA #FIXED "0.1">
<!ATTLIST osm_traffic generator          CDATA #IMPLIED>

<!ELEMENT traffic_api (version, (tile_num |tile_deg), waynodes, (wayref, noderef?)?) >

<!ELEMENT version EMPTY>
<!ATTLIST version minimum        CDATA #REQUIRED>
<!ATTLIST version maximum        CDATA #REQUIRED>

<!ELEMENT tile_num EMPTY>
<!ATTLIST tile_num level         CDATA #REQUIRED>
<!ATTLIST tile_num x             CDATA #REQUIRED>
<!ATTLIST tile_num y             CDATA #REQUIRED>

<!ELEMENT tile_deg EMPTY>
<!ATTLIST tile_deg level         CDATA #REQUIRED>
<!ATTLIST tile_deg lat           CDATA #REQUIRED>
<!ATTLIST tile_deg lon           CDATA #REQUIRED>

<!ELEMENT waynodes EMPTY>
<!ATTLIST waynodes maximum       CDATA #REQUIRED>

<!ELEMENT noderef EMPTY>
<!ATTLIST noderef id             CDATA #REQUIRED>

<!ELEMENT wayref EMPTY>
<!ATTLIST wayref id              CDATA #REQUIRED>

and the used XML file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE osm_traffic SYSTEM "update.dtd">

<osm_traffic version="0.1" generator="test">
  <traffic_api>
    <version minimum="0.1" maximum="0.1"/>
    <tile_num level="14" x="34" y= "45"/>
    <waynodes maximum="2000"/>
    <wayref id="5000"/>  <!-- optional, but mandatory if noderef below is given -->
    <noderef id="5000"/>  <!-- optional -->
  </traffic_api>
</osm_traffic>

the generated XSD file by thw w3c converter:

<schema
  xmlns="http://www.w3.org/2000/10/XMLSchema"
  targetNamespace="http://www.w3.org/namespace/"
  xmlns:t="http://www.w3.org/namespace/">

 <element name="osm_traffic">
  <complexType>
   <sequence>
    <element ref="t:traffic_api"/>
   </sequence>
   <attribute name="generator" type="string" use="optional"/>
  </complexType>
 </element>

 <element name="traffic_api">
  <complexType>
   <sequence>
    <element ref="t:version"/>
    <choice>
     <element ref="t:tile_num"/>
     <element ref="t:tile_deg"/>
    </choice>
    <element ref="t:waynodes"/>
    <sequence minOccurs="0" maxOccurs="1">
     <element ref="t:wayref"/>
     <element ref="t:noderef" minOccurs="0" maxOccurs="1"/>
    </sequence>
   </sequence>
  </complexType>
 </element>

 <element name="version">
  <complexType>
   <attribute name="maximum" type="string" use="required"/>
  </complexType>
 </element>

 <element name="tile_num">
  <complexType>
   <attribute name="y" type="string" use="required"/>
  </complexType>
 </element>

 <element name="tile_deg">
  <complexType>
   <attribute name="lon" type="string" use="required"/>
  </complexType>
 </element>

 <element name="waynodes">
  <complexType>
   <attribute name="maximum" type="string" use="required"/>
  </complexType>
 </element>

 <element name="noderef">
  <complexType>
   <attribute name="id" type="string" use="required"/>
  </complexType>
 </element>

 <element name="wayref">
  <complexType>
   <attribute name="id" type="string" use="required"/>
  </complexType>
 </element>
</schema>

Even when manually adding the namespace xs and adding the line:

<?xml version="1.0" encoding="UTF-8"?>

at the beginning of the schema, erlsom fails with the message:

** exception error: no match of right hand side value 
                    {error,
                        [{exception,{error,"unknown tag: schema"}},
                         {stack,[undefined]},
                         {received,
                             {startElement,[],"schema","xs",
                                 [{attribute,"targetNamespace",[],[],
                                      "http://www.w3.org/namespace/"}]}}]}
     in function  erlsomtest:run/0 (src/erlsomtest.erl, line 6)

float values become strings

Hello, Willem.

I noticed, that integer values in schema become integers in response, but floats become strings (or stay strings).

XSD:

<xsd:complexType name="ItemsAvailResult">
  <xsd:all>
    <xsd:element name="No" type="xsd:string" nillable="true"/>
    <xsd:element name="PriceClient" type="xsd:float" nillable="true"/>
    <xsd:element name="PriceClient_RG" type="xsd:float" nillable="true"/>
    <xsd:element name="PriceClient_MSK" type="xsd:float" nillable="true"/>
    <xsd:element name="AvailableClient" type="xsd:float" nillable="true"/>
    <xsd:element name="AvailableClient_RG" type="xsd:int" nillable="true"/>
    <xsd:element name="AvailableClient_MSK" type="xsd:int" nillable="true"/>
    <xsd:element name="AvailableExpected" type="xsd:int" nillable="true"/>
    <xsd:element name="AvailableExpectedNext" type="xsd:int" nillable="true"/>
    <xsd:element name="DateExpectedNext" type="xsd:string" nillable="true"/>
  </xsd:all>
</xsd:complexType>
<xsd:complexType name="ArrayOfItemsAvailResult">
  <xsd:sequence>
    <xsd:element name="item" type="tns:ItemsAvailResult" minOccurs="0" maxOccurs="unbounded"/>
  </xsd:sequence>
</xsd:complexType>
<xsd:element name="getItemsAvailResponse">
  <xsd:complexType>
    <xsd:sequence>
      <xsd:element name="getItemsAvailResult" type="tns:ArrayOfItemsAvailResult"/>
    </xsd:sequence>
  </xsd:complexType>
</xsd:element>

Response:

<?xml version="1.0" encoding="UTF-8"?>
  <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="https://api.merlion.com/dl/mlservice2">
    <SOAP-ENV:Body>
      <ns1:getItemsAvailResponse>
        <getItemsAvailResult>
          <item>
            <No>47031</No>
            <PriceClient>5.9</PriceClient>
            <PriceClient_RG>5.9</PriceClient_RG>
            <PriceClient_MSK>5.9</PriceClient_MSK>
            <AvailableClient>0</AvailableClient>
            <AvailableClient_RG>0</AvailableClient_RG>
            <AvailableClient_MSK>0</AvailableClient_MSK>
            <AvailableExpected>0</AvailableExpected>
            <AvailableExpectedNext>60</AvailableExpectedNext>
            <DateExpectedNext>2014-05-14</DateExpectedNext>
          </item>
        </getItemsAvailResult>
      </ns1:getItemsAvailResponse>
    </SOAP-ENV:Body>
  </SOAP-ENV:Envelope>

Returned data by detergent:

{ok,undefined,
    [#'p:getItemsAvailResponse'{
         anyAttribs = [],
         getItemsAvailResult = 
             #'p:ArrayOfItemsAvailResult'{
                 anyAttribs = [],
                 item = 
                     [#'p:ItemsAvailResult'{
                          anyAttribs = [],'No' = "47031",'PriceClient' = "5.9",
                          'PriceClient_RG' = "5.9",'PriceClient_MSK' = "5.9",
                          'AvailableClient' = 0,'AvailableClient_RG' = 0,
                          'AvailableClient_MSK' = 0,'AvailableExpected' = 0,
                          'AvailableExpectedNext' = 60,
                          'DateExpectedNext' = "2014-05-14"}]}}]}

AvailableClient* and AvailableExpected* are int, but Price* are strings and must be floats.

Missing type crashes erlsom

While writing up the example for the other bug I stumbled across something else.

The TypeReference is a tuple and not an atom in this particular case so the error report crashes instead.

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="whatnot" type="whatnot-type" />
</xs:schema>

<whatnot/>

> erlsom:scan(element(2, file:read_file("example.xml")), element(2, erlsom:compile_xsd_file("example.xsd"))).
** exception throw: {'EXIT',
                     {error,
                      [{exception,
                        {badarg,
                         [{erlang,atom_to_list,[{'#PCDATA',char}],[]},
                          {erlsom_lib,findType,6,
                           [{file,"src/erlsom_lib.erl"},{line,451}]},
                          {erlsom_parse,stateMachine,2,
                           [{file,"src/erlsom_parse.erl"},{line,376}]},
                          {erlsom_parse,xml2StructCallback,2,
                           [{file,"src/erlsom_parse.erl"},{line,295}]},
                          {erlsom_sax_utf8,wrapCallback,2,
                           [{file,"src/erlsom_sax_utf8.erl"},{line,1364}]},
                          {erlsom_sax_utf8,parseContentLT,2,
                           [{file,"src/erlsom_sax_utf8.erl"},{line,912}]},
                          {erlsom_sax_utf8,parse,2,
                           [{file,"src/erlsom_sax_utf8.erl"},{line,196}]},
                          {erlsom,scan2,3,
                           [{file,"src/erlsom.erl"},{line,211}]}]}},
                       {stack,[undefined]},
                       {received,{startElement,[],"whatnot",[],[]}}]}}
     in function  erlsom:scan2/3 (src/erlsom.erl, line 215)

"No value provided for non-optional element"

hi,

I am trying to implement erlang-based gateway to betfair, betting exchange, which has SOAP based APi.

I am using Erlsom to parse Wsdl and soap and "detergent" to make soap calls. Detergent is basically a clone of yaws soap module.

Unfortunately, I keep running into all sort of issues, some of them seem to be related to Erlsom parsing features

e.g. the current issue, erlsom fails to parse a call with nillable arguments.

to replicate:

git clone [email protected]:RomanShestakov/erlsom_test.git
cd erlsom_test;make
erl -pa ./ebin -pa ./deps/detergent/ebin -pa ./deps/erlsom/ebin
test_case:call_api().

you should get :

exception throw: {error,"No value provided for non-optional element"}
in function erlsom_write:struct2xml/6
in call from erlsom_write:processAlternativeValue/8
in call from erlsom_write:processElementValues/7
in call from erlsom_write:struct2xml/6
in call from erlsom_write:processAlternativeValue/8
in call from erlsom_write:processElementValues/7
in call from erlsom_write:struct2xml/6
in call from erlsom_write:processAlternativeValue/8

the example in test_case.erl module is doing the following:

inits the model from priv/BFExchangeService.wsdl - this is wsdl describing bet fair exch. API (it is a copy from bet fair with one commented out namespace, <xsd:import namespace="http://schemas.xmlsoap.org/soap/encoding/"/> , because erlsom wouldn't parse it otherwise).
already generated BFExchangeService.hrl could be found in "include"
then I am calling one function from API, getAllMarkets with a single param, "header"

getAllMarkets, expects one mandatory param, header of type 'P:APIRequestHeader' and 5 other params, which are optional, and which i want to avoid specifying.

here is the desc. of GetAll MarketsReq from wsdl, it specs params as nillable.

<xsd:complexType name="GetAllMarketsReq">
xsd:complexContent
<xsd:extension base="types:APIRequest">
xsd:sequence
<xsd:element name="locale" nillable="true" type="xsd:string"/>
<xsd:element name="eventTypeIds" nillable="true" type="types:ArrayOfInt"/>
<xsd:element name="countries" nillable="true" type="types:ArrayOfCountryCode"/>
<xsd:element name="fromDate" nillable="true" type="xsd:dateTime"/>
<xsd:element name="toDate" nillable="true" type="xsd:dateTime"/>
/xsd:sequence
/xsd:extension
/xsd:complexContent

but erlsom parse seems to insist on specifying them

Am I doing something wrong or this is by design?

Regards, Roman

Serializing large XML results in huge memory consumption

Hi, we are dealing with huge XML files (around 100MB of base64-encoded data) and we are observing a huge spike in memory, up to 15GB of RAM. Here is an adaptation of erlsom_example.erl that reproduces the issue, along with a screenshot of the observer that demonstrates the memory spike.

%%% a simple example of the use of erlsom.
%%%
-module(erlsom_example).

%% user interface
-export([run/0]).

%% define records
-record('out:resultType', {anyAttribs, result}).
-record('out:errorType', {anyAttribs, errorCode, errorDescription}).

run() ->
  %% compile xsd
  {ok, ModelIn} = erlsom:compile_xsd_file(example_in_xsd(), [{prefix, "in"},
                                                             {strict, false}]),
  {ok, ModelOut} = erlsom:compile_xsd_file(example_out_xsd(),[{prefix, "out"},
                                                              {strict, false}]),

  %% parse xml
  {ok, _, _} = erlsom:scan_file(example_in_xml(), ModelIn),

  Data = crypto:strong_rand_bytes(100000000),
  Encoded = base64:encode(Data),
  Result = #'out:errorType'{errorCode = "01", errorDescription = Encoded},

  %% generate xml.
  Response = #'out:resultType'{result=Result},
  {ok, XmlResult} = erlsom:write(Response, ModelOut),
  io:format("Result size: ~p~n", [string:length(XmlResult)]),
  ok.


%% this is just to make it easier to test this little example
example_in_xsd() -> filename:join([codeDir(), "example_in.xsd"]).
example_out_xsd() -> filename:join([codeDir(), "example_out.xsd"]).
example_in_xml() -> filename:join([codeDir(), "example_in.xml"]).
codeDir() -> filename:dirname(code:which(?MODULE)).

We tried doing some investigation, for example by removing erlsom_lib:xmlString, with no success. Do you know what may cause the issue? I know that there is a streaming parser, but not a streaming writer...

Need help to install the library in windows-OS.

I was trying to install the library in windows OS. Whenever I give the command according to yaws website in git shell

cd erlsom; chmod a+x configure; ./configure; make

I got following error.

The term './configure' is not recognized as the name of a cmdlet, function, scr
ipt file, or operable program. Check the spelling of the name, or if a path was
included, verify that the path is correct and try again.
At line:1 char:44

cd erlsom; chmod a+x configure; ./configure <<<< ; make
- CategoryInfo : ObjectNotFound: (./configure:String) [], Command
  NotFoundException
- FullyQualifiedErrorId : CommandNotFoundException

The term 'make' is not recognized as the name of a cmdlet, function, script fil
e, or operable program. Check the spelling of the name, or if a path was includ
ed, verify that the path is correct and try again.
At line:1 char:50

cd erlsom; chmod a+x configure; ./configure; make <<<<
- CategoryInfo : ObjectNotFound: (make:String) [], CommandNotFoun
  dException
- FullyQualifiedErrorId : CommandNotFoundException
  can you please help me in this issue.
  Thanks,
  Ashik

Handling of float/double values

Very new to erlsom, working with some XSD files using "double" values throughout. In order to get these to work properly I had to make the following change to erlsom:

In erlsom_lib:translateType/1 I had to add the case of handling double and float, this resulted in the following

translateType(String) ->
case String of
"integer" -> 'integer';
"int" -> 'integer';
"QName" -> 'qname';
"boolean" -> 'bool';
"double" -> 'float';
"float" -> 'float';
_Else -> 'char'
end.

This seemed to fix both the writing of the XML containing floating point fields and the data binding of the XML.

erlsom:scan doesn't properly handle subtypes when the element in question is within a choice with elements of the same basic type

Given the following XSD:

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
        <xsd:complexType name="BaseType">
                <xsd:attribute name="id" type="xsd:ID" />
        </xsd:complexType>
        <xsd:complexType name="DerivedType">
                <xsd:complexContent>
                        <xsd:extension base="BaseType">
                                <xsd:attribute name="name" type="xsd:string" />
                        </xsd:extension>
                </xsd:complexContent>
        </xsd:complexType>
        <xsd:complexType name="BagType">
                <xsd:choice>
                        <xsd:element name="column-a" type="BaseType" />
                        <xsd:element name="column-b" type="BaseType" />
                </xsd:choice>
        </xsd:complexType>
        <xsd:element name="bag" type="BagType" />
</xsd:schema>

The following XML:

<?xml version="1.0" encoding="UTF-8"?>
<bag xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
        <column-b xsi:type="DerivedType" id="blep" name="blop" />
</bag>

fails to parse, even though it ought.

I've tracked the problem down to erlsom_pass2:pass3Alternative function, which attempts to add types to disambiguate between alternatives with the same base type. Unfortunately, it fails to account for derived types that might occur.

I see no easy fix for this. I've come up with 2 solutions so far:

Quick'n'dirty way: modify pass3Alternative to clone the whole hierarchy. Modify subtype checking to search for subtypes also within specified type clones (in this case, they'd be named something like "column-a-BaseType" and "column-a-DerivedType" etc, I guess).
More proper way: make the function produce a "wrapper type" of some sort that would have one element with one alternative, and some sort of flag that tells the scanner to immediately descend into that element. In HRL files, such types would be translated to 1-record fields, ie 2-tuples: a bit finnicky, but very workable.

I'd love to hear your opinions on this rather convoluted corner case.

Version fix for tag 1.5.0

The tag 1.5.0 still has version 1.4.2 in src/erlsom.app.src

Add version tag

rebar3 has the nice new rebar.lock feature that allows users of your library to ensure that they don't get a version that has changed incompatibly. In order to enable that, your library needs branches and/or tags to aim at in the using rebar.config.

You don't need to update to rebar3 in order for using projects that DO use it to depend on your tags or branches, though erlsom builds for me in OTP 17.5 with rebar3 compile and tests successfully with rebar3 eunit.

Please add at least one version tag to your repository to enable this, and continue to add new tags as your changes qualify as new releases. The only reference to a version I can see is version 1.2.1 in vsn.mk, but that was committed six years ago, and is only used by the make-release script.

When parsing XSD annotate tag inside attributeGroup (ref) crashes parser

handling of attributegroup and extensions

Hi,

I'm encountering an error "Unexpected attribute" when using a schema which contains a combination of attributeGroup and extensions.

I have committed a failing test-case to illustrate this here: mdemars@b161953

Also, the .xml and .xsd files are attached.

NOTE: this is an extract of a larger schema, but I think this represents the issue properly.

comxerv.zip

All element not supported as child of extension

Hi,

I'm currently trying to compile the XSD definitions provided in RFC 5707. Unfortunately erlsom doesn't like all elements directly contained in extension elements. Here's an example taken from the msml-conf-core-datatypes.xsd:

  <xs:element name="modifyconference" substitutionGroup="msmlRequest">
    <xs:complexType>
      <xs:complexContent>
        <xs:extension base="msmlRequestType">
          <xs:all>
            <xs:element name="audiomix" type="audioMixType" minOccurs="0"/>
            <xs:element name="videolayout" type="videoLayoutType" minOccurs="0"/>
          </xs:all>
          <xs:attribute name="id" type="confID.datatype" use="required"/>
        </xs:extension>
      </xs:complexContent>
    </xs:complexType>
  </xs:element>

A minimal example from the RFC would consist of the two files msml-core-datatypes.xsd and msml-conf-core-datatypes.xsd (which I would have attached to the issue if that would be possible).

Adding the #alt{tag = 'xsd:all', tp = 'allType'} alternative to the extensionTypeC in erlsom_parseXsd.erl seems to solve the issue. However, I don't know if that is enough to add support for this structure.

Regards
Tobias

willemdj / erlsom Goto Github PK

erlsom's People

Stargazers

Watchers

Forkers

erlsom's Issues

Evidences

Problem

Recommend Projects

Recommend Topics

Recommend Org