tefra/xsdata

Naive XML & JSON Bindings for python

xsdata's Issues

XML Serializer Value fields

Xml serializer creates nested value element instead of simply setting the text attribute of the element

XmlParser text elements are sometimes empty

From lxml

Note that the text, tail, and children of an Element are not necessarily present yet when receiving the start event. Only the end event guarantees that the Element has been parsed completely.

Resolve remote schemas

Partial parsing

Currently the parser fails on unknown elements that can't be mapped to any class field.

The xml parser should accept a config object with a flag to skip unknown elements

Restricting Enumerations

Restricting enumeration I know is dump but doesn't work instead all attributes are appended duplicating existing

More HL7v3 enum fun -- Unknown enumerations

This one's harder to pin down, but in the Connect types repo, if you try to run (in the NE2008 directory) xsdata --package hl7 multicacheschemas/ xsdata will crash with:

xsdata.exceptions.AnalyzerError: Unknown enumeration RoleClassPassive: @[email protected]

which I can't quite figure out how to fix internally. It may have to do with the recent enum changes. I don't quite see what it's trying to do with the @ prefixed class name

Handle namespaces

XmlParser should respect default namespace inheritance

Problem

• a:soap
  • b:null -> inherit soap
  • c:null -> inherit soap
  • d:foo
    • e:null -> should inherit foo, it goes back to soap
  • f:null -> inherit soap

JsonSerializer include root element

give the option to include root element

Serializer / Parser Config

Currently the implementation are too opinionated add config classes to control the output to fit various needs

Ideas:

• trim zero decimals
• fail on unknown fields
• ignore additional fields

Recursive error: inner class extending outer class

In python inner classes are not allowed to extend outer classes

class Foo:
   pass

    Class Bar(Foo):
        pass

This is a sample that demonstrates the issue from w3c test suite msData/particles/particlesEb041.xsd

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
targetNamespace="foo" 	xmlns="foo"> 
<xsd:complexType name="foo">
		<xsd:sequence>
			<xsd:element name="foo"/>
			<xsd:element name="bar" minOccurs="0">
				<xsd:complexType>
					<xsd:complexContent>
						<xsd:restriction base="foo">
							<xsd:sequence>
								<xsd:element name="foo"/>
							</xsd:sequence>
						</xsd:restriction>
					</xsd:complexContent>
				</xsd:complexType>
			</xsd:element>
		</xsd:sequence>
</xsd:complexType>

<xsd:element name="root" type="foo" />

</xsd:schema>

Possible solutions: https://stackoverflow.com/a/39077925

Restriction inheritance

The generator is fancy and small but skips over parent restrictions while iterating over elements to find the right ones for attributes

Field metadata name should be omitted if it matches the field name

This was the default behavior but it changed at some point...

Implement XML Serializer for at the dataclass theme

XML Parser qualified attributes

chapter 13

Enums with member type and embedded types are not included properly

Great work on this, it's really a good tool.

However, consider this xsd: https://gist.github.com/nimish/ce45c437e82f9b565ca576f768af26fb

The analyzer culls the "Extension" type entirely. Something else should be done to properly include all the values as a proper Union type

Generate anonymous enumerations

Anonymous enumerations are not generated

xsi:type attribute shouldn't be allowed to have a default/fixed value

Otherwise the xml parsing/serializing causes validation errors

Integrate graphviz as a the printer's output

https://pypi.org/project/graphviz/

xs:openContent with mode suffix generates wildcard at wrong position

Reference issue with extended unions and fixed attributes

<?xml version="1.0" encoding="UTF-8" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">

    <xs:simpleType name="AddressPartType">
        <xs:union memberTypes="AdditionalLocator">
            <xs:simpleType>
                <xs:restriction base="xs:string">
                    <xs:enumeration value="DEL" />
                </xs:restriction>
            </xs:simpleType>
        </xs:union>
    </xs:simpleType>
    <xs:simpleType name="AdditionalLocator">
        <xs:restriction base="xs:string">
            <xs:enumeration value="ADL" />
        </xs:restriction>
    </xs:simpleType>

    <xs:complexType name="adxp.delimiter" mixed="true">
        <xs:attribute name="partType" type="AddressPartType" fixed="DEL"/>
	</xs:complexType>
</xs:schema>

This example schema fails to generate with:

xsdata.exceptions.AnalyzerError: Unknown enumeration AdditionalLocator: DEL

There's an additional issue, that "DEL" is considered a python stop word even if it actually wouldn't conflict, since the only cases that happens is when the casing matches too.

Simplified from the HL7v3 schemas.

xs:list inner classes doesn't work

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:simpleType name="AvailableSizesType">
    <xs:list>
      <xs:simpleType>
        <xs:restriction base="xs:token">
          <xs:enumeration value="small" />
          <xs:enumeration value="medium" />
          <xs:enumeration value="large" />
        </xs:restriction>
      </xs:simpleType>
    </xs:list>
  </xs:simpleType>
</xs:schema>

the attribute type defaults to str instead of the inner enum

Native types are marked as forward references

Cast restrictions to match attribute types

https://xsdata.readthedocs.io/en/latest/tests/defxmlschema/chapter02.html#example-2-10-product-schema-document-with-target-namespace

Combining memberTypes and simpleType

Example 10-3 generated str field instead of Union[int, Enum]

Flatten extension simple conflict with complex

Flatten extension simple conflict with complex when common type has same name with a complex type or element.

xsdata common type is anything except elements and complexTypes

<xs:attribute name="fooType"/>
	<xs:complexType name="fooType">
		<xs:sequence>
			<xs:element name="myEle3" type="xs:NMTOKEN"/>
			<xs:element name="myEle4" type="xs:long"/>
		</xs:sequence>
		<xs:attribute ref="fooType"/>
	</xs:complexType>
	<xs:element name="root" type="fooType"/>

Recursion limit for complicated XSDs

For some truly cursed XSDs -- HL7v3 for example -- this tool tries to consolidate and flatten types recursively. However, the complexity is too much and python kills it for hitting the recursion limit or it simply segfaults/hits OOM.

This is the gauntlet of XSDs, no tool at all works besides XJC/JAXB and even then they don't do the unions correctly.

cli should accept multiple definitions as attribute

Property ordering

Enable QName enumerations

Inner classes name conflict during attr type flattening

If two attributes are forward references to inner classes named with the generic name "value" only one of them is copied

xs:list support is very limited

Currently xs:list is treated as simple text, this causes especially in list of xs:qname a lot of failures.

Analyzer: mixed content with no defined wildcard

it should generate one

Replace generated relative imports

XSD List elements are not really lists

duh change the handling to treat them as simple str fields for now, that's the best solution I can think of, at least until validation epic

xs:pattern on non string types support

the w3c suite is full of tests applying patterns on decimals, integers, dates and boolean fields.

The xsd patterns is super tricky to handle and I am still debating how to tackle that problem but for now it might make sense to override those types with simple strings.

xs:union support is very limited

Integrate w3c / xsdtests

The defxmlschema book samples were really instrumental for the last couple releases, and I am quite happy with the stability of the generators and binding modules, it's about time to test again the official w3c xsd test suite

https://github.com/w3c/xsdtests

Support anonymous Unions

attributes with anonymous unions are defaulting to type string

Handle duplication of elements and attributes

Currently xsdata generated the duplicated attribute with no exceptions.

It's valid for choice containers but invalid for sequence container. Check samples 1213, 1214, 1228

Use abstract complexType as xsi:type

Unbounded sequences *headache

Ideas

• Keep a list of sequences as class metadata
• Consider unbounded sequences as attributes and generate inner classes

XmlParser fails to bind text into array wildcard when parent doesn't support mixed content

Recursion error on flattening complex source

from w3c

        schema="saxonData/Open/open027.xsd"
        instance="saxonData/Open/open027.v2.xml"

Integration tests

Most of the test cases are too isolated, I have added the xsdata-samples repo to test real life schemas and req/res xml files but as the code base grows I start forgetting stuff.

The project needs documented simplified tests cases with real schemas and outputs/inputs

Missing types should be ignored

Examples from w3c with valid instance documents

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<xs:element name="good" type="xs:integer"/>
<xs:element name="bad" type="absent" substitutionGroup="missing"/>
</xs:schema>

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

<xs:element name="good" type="xs:integer"/>

<xs:element name="bad" type="list"/>

<xs:simpleType name="list">
 <xs:list itemType="absent"/>
</xs:simpleType> 

</xs:schema>

xs:restriction doesn't add nested simpleType restirctions

	<element name="nillable2" nillable="true">
		<simpleType>
			<restriction>
				<simpleType>
					<list itemType="int" />
				</simpleType>
				<minLength value="2" />
			</restriction>
		</simpleType>
	</element>

    value: Optional[int] = field(
        default=None,
        metadata=dict(
            min_length=2.0
        )
    )

it should add min/max occurs from list

Validation error unmapped prefix with qname values

lxml has fixed prefixes for common namespaces so the serialization breaks

<Rule xmlns:xs="http://www.w3.org/2001/XMLSchema" name="xsd:element"/>

Filter missing type duplicates

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
           xmlns:vc="http://www.w3.org/2007/XMLSchema-versioning"
		   xmlns:my="http://www.examples.org">

	<xs:element name="Example" type="xs:int" vc:typeAvailable="xs:int" />
	
	<xs:element name="Example" type="my:myInt" vc:typeAvailable="xs:int my:myInt" />
	
</xs:schema>
``

Wrong default value for enum fields

The default value should be TypeType.ARC instead of the string value "arc"

class TypeType(Enum):
    """
    :cvar ARC:
    :cvar EXTENDED:
    :cvar LOCATOR:
    :cvar RESOURCE:
    :cvar SIMPLE:
    :cvar TITLE:
    """
    ARC = "arc"
    EXTENDED = "extended"
    LOCATOR = "locator"
    RESOURCE = "resource"
    SIMPLE = "simple"
    TITLE = "title"


@dataclass
class ArcType:

    type: TypeType = field(
        init=False,
        default="arc",
        metadata=dict(
            type="Attribute",
            namespace="http://www.w3.org/1999/xlink",
            required=True
        )
    )