basgys/goxml2json

XML to JSON converter written in Go (no schema, no structs)

goxml2json's Issues

Converting xml to wrong json

I used xj.Convert() to convert the following xml file(in sample.tar.gz) to json file . When I tried to unmarshal the same json file using "encoding/json", its giving me error "invalid character '\n' in string literal".

sample.tar.gz

Is this repo alive?

Is the author maintains the project or perhaps any well-known forks exist?

Use var for attrPrefix and contentPrefix

As of now we cannot remove attrPrefix and contentPrefix by setting them to a blank string as the decoder checks on that and defaults to their const attrPrefix and contentPrefix

In our case we'd like not to use any prefixes onto either of the extracted values. A simple fix could be to define attrPrefix and contentPrefix as var instead of const in order to allow overriding them using ldflags

And if i don't want any prefix ??

`func (dec *Decoder) Decode(root *Node) error {

if dec.contentPrefix == "" {
	dec.contentPrefix = contentPrefix
}
if dec.attributePrefix == "" {
	dec.attributePrefix = attrPrefix
}

`

Undefined Package

Using #23 to force array, undefined error

too many arguments in call to xml2json.Convert

	json, err := xj.Convert(reader, xj.WithNodes(
		xj.NodePlugin(CaseFileEventStatementsPath, xj.ToArray()),
		xj.NodePlugin(CaseFileOwnersPath, xj.ToArray()),
		xj.NodePlugin(CaseFileStatements, xj.ToArray()),
		xj.NodePlugin(MadridInternationalFilingRecordPath, xj.ToArray()),
		xj.NodePlugin(DesignSearchesPath, xj.ToArray()),
	))

src/utils/utils.go:101:34: undefined: xml2json.WithNodes
src/utils/utils.go:102:3: undefined: xml2json.NodePlugin
src/utils/utils.go:102:46: undefined: xml2json.ToArray
src/utils/utils.go:103:3: undefined: xml2json.NodePlugin
src/utils/utils.go:103:37: undefined: xml2json.ToArray
src/utils/utils.go:104:3: undefined: xml2json.NodePlugin
src/utils/utils.go:104:37: undefined: xml2json.ToArray
src/utils/utils.go:105:3: undefined: xml2json.NodePlugin
src/utils/utils.go:105:54: undefined: xml2json.ToArray
src/utils/utils.go:106:3: undefined: xml2json.NodePlugin

Invalid parsing of integers with one digit or the value `0`

It seems that using the xj.Int converter fails to recognize the number 0 and any single-digit number.

---

The following simple patch solves this issue:

diff --git i/jstype.go w/jstype.go
index cfbf3b7..bd08245 100644
--- i/jstype.go
+++ w/jstype.go
@@ -45,23 +45,23 @@ func isFloat(s string) bool {
 		_, err := strconv.ParseFloat(s, 64)
 		if err == nil { // the string successfully converts to a decimal
 			output = true
 		}
 	}
 	return output
 }
 
 func isInt(s string) bool {
 	var output = false
-	if len(s) > 1 {
+	if len(s) >= 1 {
 		_, err := strconv.Atoi(s)
 		if err == nil { // the string successfully converts to an int
-			if s[0] == '0' {
+			if s != "0" && s[0] == '0' {
 				// if the first rune is '0' and there is more than 1 rune, then the input is most likely a float or intended to be
 				// a string value -- such as in the case of a guid, or an international phone number
 			} else {
 				output = true
 			}
 		}
 	}
 	return output
 }

New version?

Hi!,

I've seen this interesting PR.

Could you create a new version of the last corrections? I'm using glide and want to keep semver packages.

Or tell us if the project is abandoned to find other solutions or create a new forked one.

Thanks a lot for your work!

Yu.

Parsing invalid XML returns `""` instead of error

It seems that trying to parse the following invalid XML does not yield an error, but instead it returns ""\n:

<test> < <a>1</a> <b>2</b> <c>3</c> <d>4/d> </test>

Parse xml with CDATA failed

Possible to force array?

Hi, is it possible for force an array? For example i have the following xml:

<?xml version="1.0" encoding="UTF-8"?>
<rss>
    <channel>
        <item>
            <apps>
                <app>App One</app>
            </apps>
        </item>
        <item>
            <apps>
                <app>App Two</app>
                <app>App Three</app>
            </apps>
        </item>
    </channel>
</rss>

which converts to

{
  "rss": {
    "channel": {
      "item": [
        { "apps": { "app": "App One" } },
        { "apps": { "app": ["App Two", "App Three"] } }
      ]
    }
  }
}

The problem is that the Apps/app node is an object but the second & third Apps/app node is an array. I understand why given the structure, but is it possible to force the first node to also be an array?
Tks,
Gary

Conversion do not happen if xml encoding is "ISO-8859-1"

Works when encoding is "UTF-8"

Parsing returns empty string without error. Invalid character entity

Version Used: 1.1.0
XML to Parse:
<msg_content>XML start 04/06/22 09:24 987654321&#13;&#10;&#13;&#10;&#13;&#10;&#13;&#10;Thank you,&#13;&#10;&#13;&#10;O:&Acirc;&#160; 13 Ext. 20601&#13;&#10;M: 12&#13;&#10;Sx:&Acirc;&#160; qw&#13;&#10;Cl:&Acirc;&#160; ui&#13;&#10;K:&Acirc;&#160; [email protected]&#13;&#10;&#13;&#10;&Acirc;&#160; &#13;&#10;&#13;&#10; &Acirc;&#160; &#13;&#10;&Acirc;&#160;&Acirc;&#160;&Acirc;&#160;&Acirc;&#160; &#13;&#10;End of message XML</msg_content>
Troubleshooting steps :
Used golang's defauld xml decoder. error: XML syntax error on line 1: invalid character entity Â

Add support for `null` custom JSType

Sometimes an empty XML tag should be interpreted as null.

The following patch add support for treating as null the following:

• the empty string;
• the string null and the string NULL;

diff --git i/jstype.go w/jstype.go
index bd08245..3efdc43 100644
--- i/jstype.go
+++ w/jstype.go
@@ -7,35 +7,38 @@ import (
 
 // https://cswr.github.io/JsonSchema/spec/basic_types/
 // JSType is a JavaScript extracted from a string
 type JSType int
 
 const (
 	Bool JSType = iota
 	Int
 	Float
 	String
+	Null
 )
 
 // Str2JSType extract a JavaScript type from a string
 func Str2JSType(s string) JSType {
 	var (
 		output JSType
 	)
 	s = strings.TrimSpace(s) // santize the given string
 	switch {
 	case isBool(s):
 		output = Bool
 	case isFloat(s):
 		output = Float
 	case isInt(s):
 		output = Int
+	case isNull(s):
+		output = Null
 	default:
 		output = String // if all alternatives have been eliminated, the input is a string
 	}
 	return output
 }
 
 func isBool(s string) bool {
 	return s == "true" || s == "false"
 }
 
@@ -58,10 +61,19 @@ func isInt(s string) bool {
 			if s != "0" && s[0] == '0' {
 				// if the first rune is '0' and there is more than 1 rune, then the input is most likely a float or intended to be
 				// a string value -- such as in the case of a guid, or an international phone number
 			} else {
 				output = true
 			}
 		}
 	}
 	return output
 }
+
+func isNull(s string) bool {
+	switch s {
+		case "", "null", "NULL" :
+			return true
+		default :
+			return false
+	}
+}
diff --git i/plugins.go w/plugins.go
index 6b93ffe..25937f4 100644
--- i/plugins.go
+++ w/plugins.go
@@ -45,21 +45,23 @@ func (tc *customTypeConverter) AddTo(e *Encoder) *Encoder {
 	e.tc = tc
 	return e
 }
 
 func (tc *customTypeConverter) Convert(s string) string {
 	// remove quotes if they exists
 	if strings.HasPrefix(s, `"`) && strings.HasSuffix(s, `"`) {
 		s = s[1 : len(s)-1]
 	}
 	jsType := Str2JSType(s)
-	if tc.parseAsString(jsType) {
+	if jsType == Null {
+		s = "null"
+	} else if tc.parseAsString(jsType) {
 		// add the quotes removed at the start of this func
 		s = `"` + s + `"`
 	}
 	return s
 }
 
 // WithAttrPrefix appends the given prefix to the json output of xml attribute fields to preserve namespaces
 func WithAttrPrefix(prefix string) *attrPrefixer {
 	ap := attrPrefixer(prefix)
 	return &ap

Open to changes to preserving whitespace and sanitizing strings?

Hey,

I'd like to contribute two changes upstream but, before I do, I wanted to get your thoughts on if you think you'd accept the changes.

Change #1 - Preserve Whitespace

I'd like to update https://github.com/basgys/goxml2json/blob/master/decoder.go#L136 to not trim white space. I was thinking, to preserve backwards-compatibility I do the following:

1. Update trimNonGraphic to be a function receiver of *Decoder
2. Update Decoder to have a doNotTrimWhitespace property
3. Add a SetDoNotTrimWhitespace *Decoder` function receiver

Change #2 - Customize encoding string sanitization

According to the accepted answer of https://stackoverflow.com/questions/3020094/how-should-i-escape-strings-in-json the only characters we must escape is \, ", and control codes (U+0020 and below). The library currently also escapes <, >, and &. I have an application that needs & to remain unescaped. I was hoping we could allow for users to customize which characters are escaped while retaining backwards-compatibility with the following changes to https://github.com/basgys/goxml2json/blob/master/encoder.go:

1. Update Encoder with a new charactersToEscape property. If not set, default to <>&
2. Update https://github.com/basgys/goxml2json/blob/master/encoder.go#L122 to remove <, >, and & specifically and instead check if the b character is contained within the encoder's charactersToEscape. Would also require updating sanitiseString to be a function receiver of *Encoder.

Thoughts? With the changes everything would remain the same (by default) but still allow for folks to customize things as needed.

can't uses xj.WithTypeConverter and xj.Float

	xml := strings.NewReader(`<?xml version="1.0" encoding="UTF-8"?><price>19.95</price>`)
	json, err := xj.Convert(xml, xj.WithTypeConverter(xj.Float))
	if err != nil {
		panic("That's embarrassing...")
	}
	fmt.Println(json.String())
	// {"price": 19.95}

why i can't uses xj.WithTypeConverter and xj.Float

and how can i may
<?xml version="1.0" encoding="UTF-8"?><name>'Tom'</name> <price>19.95</price>)
to json :
{"name:"Tom",price": 19.95}
name is string , price is number

Remove printing of debugging information to `stdout`

It seems that a fmt.Println used in debugging reached into the committed code. As a side-effect this breaks applications that want to use stdout for inter-process communication.

---

The simple patch bellow solves this issue:

diff --git i/encoder.go w/encoder.go
index be69046..e697fe1 100644
--- i/encoder.go
+++ w/encoder.go
@@ -1,15 +1,14 @@
 package xml2json
 
 import (
 	"bytes"
-	"fmt"
 	"io"
 	"unicode/utf8"
 )
 
 // An Encoder writes JSON objects to an output stream.
 type Encoder struct {
 	w               io.Writer
 	err             error
 	contentPrefix   string
 	attributePrefix string
@@ -95,21 +94,20 @@ func (enc *Encoder) format(n *Node, lvl int) error {
 			}
 			i++
 		}
 
 		enc.write("}")
 	} else {
 		s := sanitiseString(n.Data)
 		if enc.tc == nil {
 			// do nothing
 		} else {
-			fmt.Println(s)
 			s = enc.tc.Convert(s)
 		}
 		enc.write(s)
 
 	}
 
 	return nil
 }
 
 func (enc *Encoder) write(s string) {

How to add dynamic NodePlugin in Plugin

        if IsArrayTrue {
		xj.NodePlugin("x.y", xj.ToArray()) // how to add this line to xj.WithNodes(...)
	}

	jsn, err := xj.Convert(bytes.NewReader(xmlByte), xj.WithNodes(
		xj.NodePlugin("foo.bar", xj.ToArray()),
	)