Giter Site home page Giter Site logo

alexejk / go-xmlrpc Goto Github PK

View Code? Open in Web Editor NEW
18.0 3.0 7.0 165 KB

An XML-RPC Client for Go

Home Page: https://alexejk.io/article/handling-xmlrpc-in-go/

License: MIT License

Dockerfile 0.71% Makefile 2.46% Shell 3.46% Go 93.37%
xml-rpc xml-rpc-client

go-xmlrpc's Introduction

XML-RPC Client for Go

This is an implementation of client-side part of XML-RPC protocol in Go.

GitHub Workflow Status codecov Go Report Card

GoDoc GitHub GitHub release (latest SemVer)

Usage

Add dependency to your project:

go get -u alexejk.io/go-xmlrpc

Use it by creating an *xmlrpc.Client and firing RPC method calls with Call().

package main

import(
    "fmt"

    "alexejk.io/go-xmlrpc"
)

func main() {
    client, _ := xmlrpc.NewClient("https://bugzilla.mozilla.org/xmlrpc.cgi")
    defer client.Close()
	
    result := &struct {
        BugzillaVersion struct {
            Version string
        }
    }{}

    _ = client.Call("Bugzilla.version", nil, result)
    fmt.Printf("Version: %s\n", result.BugzillaVersion.Version)
}

Customization is supported by passing a list of Option to the NewClient function. For instance:

  • To customize any aspect of http.Client used to perform requests, use HttpClient option, otherwise http.DefaultClient will be used
  • To pass custom headers, make use of Headers option.
  • To not fail parsing when unmapped fields exist in RPC responses, use SkipUnknownFields(true) option (default is false)

Argument encoding

Arguments to the remote RPC method are passed on as a *struct. This struct is encoded into XML-RPC types based on following rules:

  • Order of fields in struct type matters - fields are taken in the order they are defined on the type.
  • Numbers are to be specified as int (encoded as <int>) or float64 (encoded as <double>)
  • Both pointer and value references are accepted (pointers are followed to actual values)

Response decoding

Response is decoded following similar rules to argument encoding.

  • Order of fields is important.
  • Outer struct should contain exported field for each response parameter (it is possible to ignore unknown structs with SkipUnknownFields option).
  • Structs may contain pointers - they will be initialized if required.
  • Structs may be parsed as map[string]any, in case struct member names are not known at compile time. Map keys are enforced to string type.

Handling of Empty Values

If XML-RPC response contains no value for well-known data-types, it will be decoded into the default "empty" values as per table below:

XML-RPC Value Default Value
<string/> ""
<int/>, <i4/> 0
<boolean/> false
<double/> 0.0
<dateTime.iso8601/> time.Time{}
<base64/> nil
<array><data/><array> nil

As per XML-RPC specification, <struct> may not have an empty list of <member> elements, thus no default "empty" value is defined for it. Similarly, <array/> is considered invalid.

Field renaming

XML-RPC specification does not necessarily specify any rules for struct's member names. Some services allow struct member names to include characters not compatible with standard Go field naming. To support these use-cases, it is possible to remap the field by use of struct tag xmlrpc.

For example, if a response value is a struct that looks like this:

<struct>
    <member>
        <name>stringValue</name>
        <value><string>bar</string></value>
    </member>
    <member>
        <name>2_numeric.Value</name>
        <value><i4>2</i4></value>
    </member>
</struct>

it would be impossible to map the second value to a Go struct with a field 2_numeric.Value as it's not valid in Go. Instead, we can map it to any valid field as follows:

v := &struct {
    StringValue string
    SecondNumericValue string `xmlrpc:"2_numeric.Value"`
}{}

Similarly, request encoding honors xmlrpc tags.

Building

To build this project, simply run make all. If you prefer building in Docker instead - make build-in-docker is your friend.

go-xmlrpc's People

Contributors

alexejk avatar dependabot-preview[bot] avatar dependabot[bot] avatar nightapes avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

go-xmlrpc's Issues

Passing arguments to the call

Is there any documentation and example how to pass arguments to the call function, should it be struct or interface?

Unmarshal to struct

Hi,

I'm trying to unmarshal some xmlrpc response to some go struct without any success 😅

I'm always end up with either:

  • failed decoding array item at index 2: invalid field type: expected 'struct' or 'map', got 'interface'
  • invalid field type: expected 'slice', got 'map'
  • or even fatal failure

Currently, I got to this:

var _payload = `
<methodResponse>
  <params>
    <param>
      <value>
        <array>
          <data>
            <value>
              <i4>200</i4>
            </value>
            <value>OK</value>
            <value>
              <struct>
                <member>
                  <name>status</name>
                  <value>OK</value>
                </member>
                <member>
                  <name>contact</name>
                  <value>&lt;sip:[email protected]:5060&gt;;expires=60</value>
                </member>
              </struct>
            </value>
          </data>
        </array>
      </value>
    </param>
  </params>
</methodResponse>`

func TestNewXMLRPC(t *testing.T) {
	resp, err := xmlrpc.NewResponse([]byte(_payload))

	t.Logf("%+v\n", resp.Params)

	require.NoError(t, err)
	dcd := &_xmlrpc.StdDecoder{}
	out := []struct{}{} // any{}
	// struct {
	// 		Test []any // map[string]any //  struct {
	// 		// 	Ret int
	// 		// }
	// 		// // Str []struct {
	// 		// // 	Int int
	// 		// // }
	// 	}{}

	require.NoError(t, dcd.Decode(resp, &out))
	require.Equal(t, &struct{}{}, &out)

	// require.
}

A lil help please ? thanks ❤️

Error: reading body number of exported fields (3) on response type doesnt match expectation (1)

First of all thank you very much for your great work. I have an weird XML-RPC server implementation which does response with a different amount of values in case of an internal error occurred, hence the message

Error: reading body number of exported fields (3) on response type doesn't match expectation (1)

Is there an option to omit the struct value in case it is empty? Maybe something like this:

result := &struct {
     Error   int                                                                                                                                                            
     Version string `xmlrpc:"Version,omitempty"`                                                                                                                              
     Type    string `xmlrpc:"Type,omitempty"`
}{}

In case of a successful operation the returned data from the device is {0,"4711","AA"} and in case of an error only {-100} is returned.

Possible goroutines leak

Hi!

I found some issue while using this package. Seems NewClient creation leads to goroutines leak.
Could be reproduced on the simple request:

package main

import (
	"fmt"
	"runtime"

	"alexejk.io/go-xmlrpc"
)

func request() {
	client, err := xmlrpc.NewClient("https://bugzilla.mozilla.org/xmlrpc.cgi")
	if err != nil {
		panic(err)
	}
	defer client.Close()
	result := &struct {
		BugzillaVersion struct {
			Version string
		}
	}{}
	_ = client.Call("Bugzilla.version", nil, result)
	fmt.Printf("Version: %s\n", result.BugzillaVersion.Version)
}

func main() {
	for i := 0; i < 10; i++ {
		request()
		fmt.Printf("NumGoroutine: %d\n", runtime.NumGoroutine())
	}
}

Output:

Version: 20220119.1
NumGoroutine: 3
Version: 20220119.1
NumGoroutine: 4
Version: 20220119.1
NumGoroutine: 5
Version: 20220119.1
NumGoroutine: 6
Version: 20220119.1
NumGoroutine: 7
Version: 20220119.1
NumGoroutine: 8
Version: 20220119.1
NumGoroutine: 10
Version: 20220119.1
NumGoroutine: 10
Version: 20220119.1
NumGoroutine: 11
Version: 20220119.1
NumGoroutine: 12

I tried to find the root cause myself, but have no results yet.

Your .dependabot/config.yml contained invalid details

Dependabot encountered the following error when parsing your .dependabot/config.yml:

The property '#/update_configs/0/update_schedule' value "live" did not match one of the following values: daily, weekly, monthly

Please update the config file to conform with Dependabot's specification using our docs and online validator.

Support for decoding to aliased types

Given a struct response with e.g a string member, it should be possible to parse that member into a type that is an alias of a string.

We should check if field is assignable or convertible. If conversion is possible - convert prior to assignment.

Support decoding to pointer of struct

Currently if response value contains a pointer to struct, we get the following error:

invalid field type: expected 'struct', got 'ptr'

This is not convenient and doesn't let us use pointers in response objects without additional processing.

How are the parameters sorted?

<?xml version="1.0"?>
<methodCall>
<methodName>login</methodName>
<params>
	<param><value><string><![CDATA[username]]></string></value></param>
	<param><value><string><![CDATA[apiKey]]></string></value></param>
</params>
</methodCall>

This request contains two string value.

	err = rpcClient.Call("login", &struct {
		UserName string
		ApiKey   string
	}{
		UserName: "username",
		ApiKey:   "apiKey",
	}, nil)

i got

<methodCall>
<methodName>login</methodName>
<params>
	<param><value><string>apiKey</string></value></param>
	<param><value><string>username</string></value></param>
</params>
</methodCall>

username should be the first param .

Unable to decode empty strings properly

On a response like

<?xml version="1.0"?>
<methodResponse>
	<params>
		<param>
			<value>
				<struct>
					<member>
						<name>somestring</name>
						<value>
							<string></string>
						</value>
					</member>
				</struct>
			</value>
		</param>
	</params>
</methodResponse>

I expect somestring to be "" but I get the raw xml instead

Test case output:

--- FAIL: TestStdDecoder_DecodeRaw (0.00s)
    --- FAIL: TestStdDecoder_DecodeRaw/struct_response_empty (0.00s)
        /home/vtan/repos/go-xmlrpc/decode_test.go:90: 
            	Error Trace:	/home/vtan/repos/go-xmlrpc/decode_test.go:90
            	Error:      	Not equal: 
            	            	expected: &struct { Struct struct { Somestring string "xmlrpc:\"somestring\" json:\"somestring\"" } }{Struct:struct { Somestring string "xmlrpc:\"somestring\" json:\"somestring\"" }{Somestring:""}}
            	            	actual  : &struct { Struct struct { Somestring string "xmlrpc:\"somestring\" json:\"somestring\"" } }{Struct:struct { Somestring string "xmlrpc:\"somestring\" json:\"somestring\"" }{Somestring:"\n\t\t\t\t\t\t\t<string></string>\n\t\t\t\t\t\t"}}
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -2,3 +2,3 @@
            	            	  Struct: (struct { Somestring string "xmlrpc:\"somestring\" json:\"somestring\"" }) {
            	            	-  Somestring: (string) ""
            	            	+  Somestring: (string) (len=32) "\n\t\t\t\t\t\t\t<string></string>\n\t\t\t\t\t\t"
            	            	  }
            	Test:       	TestStdDecoder_DecodeRaw/struct_response_empty
FAIL
FAIL	alexejk.io/go-xmlrpc	0.002s
FAIL

Repro case in vtan-fortinet@f7513b3

Failure to produce request with a []any as params

I cannot successfully produce the following request payload:

<?xml version="1.0"?>
<methodCall>
  <methodName>di</methodName>
  <params>
    <param><value><string>abc</string></value></param>
    <param><value><string>def</string></value></param>
    <param><value><string>hij</string></value></param>
  </params>
</methodCall>
Attempt to use a struct
if err := ec.Encode(os.Stdout, "di", struct{ Params []string }{Params: []string{"abc", "def", "hij"}}); err != nil {
		log.Fatalf("xmlrpc encode: %v", err)
	}

Produce an invalid payload

<methodCall>
  <methodName>di</methodName>
  <params>
    <param><value><array><data>
      <value><string>webconference</string></value>
      <value><string>listRooms</string></value>
      <value><string>verysecret</string></value>
    </data></array></value></param>
  </params>
</methodCall>
Attempt to use an array

Produce a fatal

if err := ec.Encode(os.Stdout, "di", []any{"abc", "def", "hij"}); err != nil {
		log.Fatalf("xmlrpc encode: %v", err)
	}
-->
<methodCall><methodName>di</methodName>panic: reflect: call of reflect.Value.NumField on slice Value

goroutine 1 [running]:
reflect.flag.mustBe(...)
	/home/master/repo/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-amd64/src/reflect/value.go:233
reflect.Value.NumField({0x6a0300?, 0xc00018e000?, 0x497c2a?})
	/home/master/repo/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-amd64/src/reflect/value.go:2127 +0x85
alexejk.io/go-xmlrpc.(*StdEncoder).encodeArgs(0xc00008fe7f, {0x78b0a0, 0xc000098020}, {0x6a0300?, 0xc00018e000?})
	/home/master/repo/go/pkg/mod/alexejk.io/go-xmlrpc@v0.5.2/encode.go:37 +0x105
alexejk.io/go-xmlrpc.(*StdEncoder).Encode(0xc00008fe7f, {0x78b0a0, 0xc000098020}, {0x706aa3?, 0x0?}, {0x6a0300, 0xc00018e000})
	/home/master/repo/go/pkg/mod/alexejk.io/go-xmlrpc@v0.5.2/encode.go:24 +0xae
main.main()
	/home/master/repo/sbc/src/common/api/gopi/test/xmlrpc.go:36 +0x16c

Error handling

Is there any solution to handle errors? I want to get specific error codes after making a call. In my case to check for 401 (unauthorized) and update the tokens accordingly.
@alexejk

How to decode a struct of arbitrary fields?

Hi, I'm struggling to decode an XMLRPC example where there are is a struct with a number of members whose names are arbitrary, so I can't put them into a tag.

Example XML
<methodResponse>
  <params>
    <param>
      <value>
        <struct>
          <member>
            <name>TESTING1</name>
            <value>
              <array>
                <data>
                  <value>
                    <array>
                      <data>
                        <value>
                          <struct>
                            <member><name>id</name><value><string>1009470</string></value></member>
                            <member><name>pub_date</name><value><string>2020-01-11 00:00:00</string></value></member>
                            <member><name>title</name><value><string>TITLE</string></value></member>
                        </struct></value>
                        <value><struct>
                            <member><name>title</name><value><string>TITLE2</string></value></member>
                            <member><name>pub_date</name><value><string>2020-01-11 00:00:00</string></value></member>
                            <member><name>id</name><value><string>1009879</string></value></member>
                        </struct></value>
                        <value><struct>
                            <member><name>title</name><value><string>Title3</string></value></member>
                            <member><name>pub_date</name><value><string>2020-01-13 17:16:49</string></value></member>
                            <member><name>id</name><value><string>1304451</string></value></member>
                        </struct></value>
                  </data></array></value>
            </data></array></value>
          </member>
          <member>
            <name>TESTING2</name>
            <value>
              <array>
                <data>
                  <value>
                    <array>
                      <data>
                        <value>
                          <struct>
                            <member><name>id</name><value><string>1329812</string></value></member>
                            <member><name>pub_date</name><value><string>2020-01-11 00:00:00</string></value></member>
                            <member><name>title</name><value><string>NewTitle</string></value></member>
                        </struct></value>
                        <value><struct>
                            <member><name>title</name><value><string>NextTitle</string></value></member>
                            <member><name>pub_date</name><value><string>2021-01-11 00:00:00</string></value></member>
                            <member><name>id</name><value><string>1489372</string></value></member>
                        </struct></value>
                        <value><struct>
                            <member><name>title</name><value><string>Title12</string></value></member>
                            <member><name>pub_date</name><value><string>2020-01-13 17:16:49</string></value></member>
                            <member><name>id</name><value><string>1229276</string></value></member>
                        </struct></value>
                  </data></array></value>
            </data></array></value>
          </member>
        </struct>
      </value>
    </param>
  </params>
</methodResponse>

The following structure can be successfully decoded into:

type DataType struct {
	Id      string `json:"id" xmlrpc:"id"`
	PubDate string `json:"pub_date" xmlrpc:"pub_date"`
	Title   string `json:"title" xmlrpc:"title"`
}

type Data struct {
	Struct struct {
		TESTING1 [][]DataType
		TESTING2 [][]DataType
	}
}

However, I cannot know the names where TESTING1 and TESTING2 are. They will change at runtime and the dataset will get added to and removed from, so those values are essentially arbitrary and therefore cannot be written directly into the struct. I also don't have control over the RPC API, so I cannot change the structure.

Any help would be greatly appreciated.

Non Struct Response caused Panic

I'm working with an XMLRPC API that has some quirks, it has two APIs one which returns a top level struct with an unknown set of members (because it's implementing a map type as a struct with variable members) and another which returns an array of structs.

In both cases, attempting to use this library causes a panic:

package main

import (
	"fmt"

	"alexejk.io/go-xmlrpc"
)

func main() {
	client, _ := xmlrpc.NewClient("https://pypi.org/pypi")

	result := &[]struct {
		Name      string
		Version   string
		Timestamp string
		Action    string
		Serial    uint64
	}{}

	_ = client.Call("changelog_since_serial", &struct{ SinceSerial int }{SinceSerial: 22292800}, result)
	fmt.Printf("Items: %d\n", len(*result))
}
panic: reflect: call of reflect.Value.NumField on slice Value

goroutine 19 [running]:
reflect.flag.mustBe(...)
        /home/linuxbrew/.linuxbrew/Cellar/go/1.21.6/libexec/src/reflect/value.go:233
reflect.Value.NumField({0x68e2a0?, 0xc00011c498?, 0x20000?})
        /home/linuxbrew/.linuxbrew/Cellar/go/1.21.6/libexec/src/reflect/value.go:2062 +0x85
alexejk.io/go-xmlrpc.fieldsMustEqual({0x68e2a0?, 0xc00011c498?}, 0x1)
        /home/dstufft/go/pkg/mod/alexejk.io/[email protected]/decode.go:305 +0xf6
alexejk.io/go-xmlrpc.(*StdDecoder).Decode(0x0?, 0xc00051d460, {0x68e2a0, 0xc00011c498})
        /home/dstufft/go/pkg/mod/alexejk.io/[email protected]/decode.go:46 +0x45
alexejk.io/go-xmlrpc.(*Codec).ReadResponseBody(0x6a3380?, {0x68e2a0?, 0xc00011c498?})
        /home/dstufft/go/pkg/mod/alexejk.io/[email protected]/codec.go:171 +0x3d
net/rpc.(*Client).input(0xc0001225a0)
        /home/linuxbrew/.linuxbrew/Cellar/go/1.21.6/libexec/src/net/rpc/client.go:141 +0x291
created by net/rpc.NewClientWithCodec in goroutine 1
        /home/linuxbrew/.linuxbrew/Cellar/go/1.21.6/libexec/src/net/rpc/client.go:206 +0xb6
exit status 2

or

package main

import (
	"fmt"

	"alexejk.io/go-xmlrpc"
)

func main() {
	client, _ := xmlrpc.NewClient("https://pypi.org/pypi")

	result := &map[string]int{}

	_ = client.Call("list_packages_with_serial", nil, result)
	fmt.Printf("Items: %d\n", len(*result))
}
panic: reflect: call of reflect.Value.NumField on map Value

goroutine 6 [running]:
reflect.flag.mustBe(...)
        /home/linuxbrew/.linuxbrew/Cellar/go/1.21.6/libexec/src/reflect/value.go:233
reflect.Value.NumField({0x6a2380?, 0xc000088528?, 0x301e000?})
        /home/linuxbrew/.linuxbrew/Cellar/go/1.21.6/libexec/src/reflect/value.go:2062 +0x85
alexejk.io/go-xmlrpc.fieldsMustEqual({0x68e240?, 0xc000088528?}, 0x1)
        /home/dstufft/go/pkg/mod/alexejk.io/[email protected]/decode.go:305 +0xf6
alexejk.io/go-xmlrpc.(*StdDecoder).Decode(0x0?, 0xc0004ac000, {0x68e240, 0xc000088528})
        /home/dstufft/go/pkg/mod/alexejk.io/[email protected]/decode.go:46 +0x45
alexejk.io/go-xmlrpc.(*Codec).ReadResponseBody(0x6a32e0?, {0x68e240?, 0xc000088528?})
        /home/dstufft/go/pkg/mod/alexejk.io/[email protected]/codec.go:171 +0x3d
net/rpc.(*Client).input(0xc0000b2600)
        /home/linuxbrew/.linuxbrew/Cellar/go/1.21.6/libexec/src/net/rpc/client.go:141 +0x291
created by net/rpc.NewClientWithCodec in goroutine 1
        /home/linuxbrew/.linuxbrew/Cellar/go/1.21.6/libexec/src/net/rpc/client.go:206 +0xb6
exit status 2

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.