mxmcherry/openrtb

Go (Golang) OpenRTB 2.5, 3.0 / AdCOM 1.0 / Native 1.2 types/enums

ARCHIVED, use prebid/openrtb instead

Prebid agreed to take over this project, so use their fork github.com/prebid/openrtb instead.

openrtb

OpenRTB, AdCOM and OpenRTB Dynamic Native Ads types for Go programming language

• openrtb2 - OpenRTB 2.5, 2.6
• openrtb3 - OpenRTB 3.0 (can lag behind because official spec is constantly updated without version bump, feel free to PR)
• adcom1 - AdCOM 1.0 (can lag behind because official spec is constantly updated without version bump, feel free to PR)
• native1 - OpenRTB Dynamic Native Ads API 1.2

Requires Go 1.13+

This library is switched to Go modules (tl;dr) as of v14.0.0, so it requires Go 1.11+ (older Go versions are not capable of using versioned paths).

Also, test/matcher library relies on newer Go error handling approach, so tests require Go 1.13+.

Using

go get -u "github.com/mxmCherry/openrtb/v17/..."

import (
	openrtb2 "github.com/mxmCherry/openrtb/v17/openrtb2"

	openrtb3 "github.com/mxmCherry/openrtb/v17/openrtb3"
	adcom1 "github.com/mxmCherry/openrtb/v17/adcom1"

	native1 "github.com/mxmCherry/openrtb/v17/native1"
	nreq "github.com/mxmCherry/openrtb/v17/native1/request"
	nres "github.com/mxmCherry/openrtb/v17/native1/response"
)

This repo follows semver - see releases. Master always contains latest code, so better use some package manager to vendor specific version.

Guidelines

Naming convention

• UpperCamelCase
• Capitalized abbreviations (e.g., AT, COPPA, PMP etc.)
• Capitalized ID keys
• Enum items with versions should include minor/patch zeros, i.e. "Foo 1.0" -> Foo10 (and not just Foo1), "Foo 1.1" -> Foo11 etc

Types

• Key types should be chosen according to OpenRTB specification (attribute types)
• Numeric types:
  • int8 - short enums (with values <= 127), boolean-like attributes (like BidRequest.test)
  • int64 - other integral types
  • float64 - coordinates, prices etc.
• Enums:
  • all enums, described in section 5, must be typed with section name singularized (e.g., "5.2 Banner Ad Types" -> type BannerAdType int8)
  • all typed enums must have constants for each element, prefixed with type name (e.g., "5.2 Banner Ad Types - XHTML Text Ad (usually mobile)" -> const BannerAdTypeXHTMLTextAd BannerAdType = 1)
  • never use iota for enum constants
  • OpenRTB (2.x) section "5.1 Content Categories" should remain untyped and have no constants

Pointers/omitempty

Pointer	Omitempty	When to use	Example
no	no	required in spec	Audio.mimes
yes	yes	required in spec, but is a part of mutually-exclusive group	Imp.{banner,video,audio,native}
no	yes	zero value ("", 0) is useless / has no meaning	Device.ua
yes	yes	zero value ("", 0) or value absence (null) has special meaning	Device.{dnt,lmt}

Using both pointer and omitempty is mostly just to save traffic / generate more "canonical" (strict) JSON.

Documentation (pkg.go.dev)

• Godoc: documenting Go code
• Each entity (type, struct key or constant) should be documented
• Ideally, copy-paste descriptions as-is, but feel free to omit section numbers, so just <GoTypeName> defines <copy-pasted description from spec>

Code organization

• Each RTB type should be kept in its own file, named after type
• File names are in underscore_case, e.g., type BidRequest should be declared in bid_request.go
• go fmt your code
• EditorConfig (not required, but useful)

openrtb's Issues

Seatbid should be empty list in no bid response

In the OpenRTB 2.5 specification,

7.1 No-Bid Signaling
This section covers best practices for using the optional no-bid signaling. See the List 5.24 for the enumerated list of no-bid reason codes.
Many exchanges support multiple response types as a no-bid:
 HTTP 204 “No Content” from the bidder (most economical in terms of bandwidth).
 AnemptyJSONobject: {}
 A well-formed no bid response:
{"id": "1234567890", "seatbid": []}
 A well-formed no bid response with a reason code:
{"id": "1234567890", "seatbid": [], "nbr": 2}

That means seatbid[] can be empty when you are returning a no bid signal with NBR.
Would it be correct to say that in https://github.com/mxmCherry/openrtb/blob/master/bid_response.go#L31-L32

It should be SeatBid []SeatBid json:"seatbid" instead of SeatBid []SeatBid json:"seatbid,omitempty"?

Question: Why did the uint*s change to int*s?

Greetings

I'm updating https://github.com/prebid/prebid-server from version 11.0.0 to the latest 14.0.0.

I'm wondering if you would please give some context for the changes in #37, when unsigned integers were changes to signed integers.

We're debating if we should add positive integer validation and am curious if you're aware of valid use cases for negative values.

Thank You

Native response asset id lost with omitempty

Hey,

if the ID of a native response asset (https://github.com/mxmCherry/openrtb/blob/master/native1/response/asset.go#L21) has a value of 0 it's lost when marshalling.

Looking through other similar issues (#33) in this repo the fix would probably to change it to a pointer?

Repo transfer

Can https://github.com/prebid take over this repo? Fork under the org, change paths and maintain it (usually it's a low-effort activity). Maybe tag it as v1 after fork (reset versioning after it gains new URL).

CC @SyntaxNode (as the latest contributor related to Prebid) for help / contact.

I'm not interested in maintaining it - I'm neither using it nor enjoying following RTB standards anymore.

I'd prefer to point to / recommend active fork (forked by active lib user) before archiving it under my account.

OpenRTB 2.5, Regs.COPPA field - make it ptr?

inside regs.go, COPPA should be a pointer so it's value of 0 doesn't get omitted when JSON marshalling?

use openrtb2.video instead of request.video in native asset

the annotation in /native1/request/video.go writes "For optional attributes please refer to OpenRTB.", i suppose the fields that list in native-specification is not enough, and it is better to use openrtb2/video.go instead of /native1/request/video.go in Assets struct.
am i right? Or the extra feilds should be added into video.ext ?

`user.keywords` field should be list (`[]string`)

user.keywords field should be list ([]string)

OpenRTB Native

Hey,

Do you plan to support OpenRTB Native objects in this project as well?

If not, we'll probably start up a separate project for it... but if you're interested, we could submit PRs here to help make it happen instead.

Proposal: separate Ext from core types

I woke up this morning with an idea which I think might be awesome. However, it'd be a fairly major change to this library... so let me know what you think.

The problem is the OpenRTB Ext fields. Since each Exchange is supposed to define their own, a core library like this is stuck typing them to be "arbitrary JSON." I think this all makes sense.

In practice, we pass these types throughout many layers of our code. So we need to Unmarshal the Ext fields once to validate input, and then decide between passing extra arguments with all our Ext structs, or else do the Unmarshal twice. Neither solution is great. It's actually a bit worse than that, because Ext means different things to different layers; we preprocess it so that one bidder can't read it and use the data to find out which other bidders are involved in the auction.

Enter.... embedded structs! I'm fairly new to Go, and have never seen a good use-case for these yet, but... I think this might be the first. Basically, it seems like we can do:

type Inner struct {
    Value int
}

type Outer struct{
    Inner
    Name string
}

func doThing(outer *Outer) {
    outer.Name // Is legal
    outer.Value // Is legal
}

So... how would you feel about decomposing the structs in this library? For example:

type CoreRequest struct {
  ... all Request fields _except_ for Ext go here ...
}

type Request struct {
  CoreRequest
  Ext    RawJSON
}

That way someone could import this library and define their own:

type MyRequest struct {
  CoreRequest
  Ext    MyStrictlyTypedExtHere
}

and get strictly-typed Ext, while still benefiting from all the other excellent spec-tracking you've done here :)

No way to omit integer fields with "omitempty" without pointer indirection

Currently there's no way to omit/not-omit fields like secure (in Impression) from JSON-representation of BidRequest regardless of the actual value (i.e. distinguish between field value and exporting the field).

Let's take Secure from Imp for example. The field in question is defined as following:

Secure int8 `json:"secure,omitempty"`

Some DSPs are unable to process this field properly due to the way they implement OpenRTB protocol, so it would be nice to omit this field from Imp altogether in such case. Or vice versa -- DSP can require the field to be present, even if it has a value of 0. However, without pointer indirection there's no way to add/remove it reliably. A value of 0 will never be exported in resulting JSON if DSP requires it ("omitempty" int8 field won't be included to JSON if it has value of 0).

So in my opinion it would be better to use pointer indirection here in all these clauses to distinguish between "empty" and "not-empty" for "omitempty" fields.

Switch `RawJSON` -> `json.RawMessage`

json.RawMessage doesn't have to be a pointer to marshal properly since Go 1.8: https://golang.org/doc/go1.8#minor_library_changes

• use json.RawMessage instead of RawJSON everywhere (incl. native/...)
• remove type RawJSON and tests for it everywhere (incl. native/...)

Better do after #20 to avoid any conflicts (though there shouldn't be any).

height and width in banner.go should not have omitempty set

Hello,

One of our demand partners has recently begin returning bid responses to us with banners where h and w are set to 0; it appears that we're discarding these fields completely, because the omitempty attribute. is set for them https://github.com/mxmCherry/openrtb/blob/master/banner.go#L26-L42

The "omitempty" option specifies that the field should be omitted from the encoding if the field has an empty value, defined as false, 0, a nil pointer, a nil interface value, and any empty array, slice, map, or string.

These fields are defined as integers in the OpenRTB2.5 spec, and so 0 is a valid value for them: https://www.iab.com/wp-content/uploads/2016/03/OpenRTB-API-Specification-Version-2-5-FINAL.pdf

We've raised this as an issue with the upstream partner, but I'm flagging this issue as well as I believe it needs to be resolved upstream from them - it's unexpected that they would set h or w to 0, but it is legal according to the spec afaict...

Cheers,

--afsheen

invalid `BAdv` comments

copy & paste error, should be Block list of advertisers by their domains (e.g., “ford.com”)..

OpenRTB 2.6

The final OpenRTB 2.6 specification was released earlier this month. I'm curious about the plans to add support in this library and if you're interested in assistance.

Improve typesafety

There are many values in the OpenRTB spec which define more narrow "legal" values than Go's native types. Some examples:

• Test mode
• AllImps
• FD

In my opinion, it'd be really nice if there were typedefs for all these values. For example, something like this:

type FD int8
const (
	EXCHANGE FD = iota
	UPSTREAM_SOURCE
)

func (fd FD) MarshalJSON() ([]byte, error) {
	switch fd {
	case EXCHANGE:
		return []byte("0"), nil
	case UPSTREAM_SOURCE:
		return []byte("1"), nil
	default:
		return nil, errors.New("FD must be 0 or 1")
	}
}

func (fd *FD) UnmarshalJSON(data []byte) error {
	var tmp int8
	if err := json.Unmarshal(data, &tmp); err != nil {
		return err
	} else {
		switch tmp {
		case 0:
			*fd = EXCHANGE
			return nil
		case 1:
			*fd = UPSTREAM_SOURCE
			return nil
		default:
			return fmt.Errorf("Cannot turn %d into an FD", tmp)
		}
	}
}

That way, the compiler would discourage people from sending invalid values in their messages. Any remaining bugs (e.g. if a caller does FD(3)) would get runtime errors when they try to marshal the objects. It also makes calling code more readable, since they'll be using your named constants.

What do you think?

protocol should be an integer and not an array of integers in video

Section 3.2.4 on OpenRTB Specification Version 2.3.1

Why the secure field in Imp be a pointer of uint8?

in 2.3.1 the spec says this field is an integer, so why just a value here instead of a pointer?

AdCom 1.0

Need AdCom 1.0 BETA implementation in github.com/mxmCherry/openrtb/adcom subpkg.

Should be completely standalone, not referencing (importing) anything else within this (or any other) repo. So branch off master, no need to wait for OpenRTB 3.0 PR #31

Add Native 1.2 Support

It looks like the latest version of OpenRTB Native 1.2 has been released: https://iabtechlab.com/wp-content/uploads/2016/07/OpenRTB-Native-Ads-Specification-Final-1.2.pdf

There are a few more fields in this version which could be added to the project.

Field "Value" in metric.go

Reference:

openrtb/metric.go

Line 28 in fd8bfbb

Value float64 `json:"value,omitempty"`

Since the "Value" field is a float between [0.0, 1.0] and and GO defaults float to 0.0, this field can not be validated if it is provided and auto-set by default.

Proposing: pointer for this field?

Any other thoughts?

consider use `json.RawMessage` for ext fields

for now, ext fields are of type interface{}, which I see two issues in production code:

• bad in decoding performance, prevent effective use of ffjson, see pquerna/ffjson#125
• read value from interface{} is not easy, sometimes we have to do conversion from interface{} to custom ext struct type

consider use ext *json.RawMesage fields to defer decoding to later stage, if there actually is a need to peek into ext fields.

how to implement the openrtb..

Hi.. im really interesting in rtb ecosystem.. can you give me suggestion how can i use openrtb i mean how to create ad tags and dashboard analytics, should i need adserver to implement ? please teach me,.

thanks

avoid []uint8

as mentioned in http://stackoverflow.com/questions/14177862/how-to-jsonize-a-uint8-slice-in-go,
[]uint8 fields are json encoded into string.

Maybe replace []uint8 with []int, or, to make things simpler, just replace all uint* with int?