Potential namespace collision?

[frictionlessdata]

Potential namespace collision?

[fils]

@pwalsh thanks for reaching out. Would love to talk to you more about this. Have a colleague also looking to expose some water column data using frictionless approaches. My data is scientific drilling cores and more geology/paleo related. Send me an email at [email protected] and lets chat about this. Would love to have a connection to Open Knowledge on this. I'm playing a role in the NSF EarthCube program too and we have a chance to advocate for data packages in that.

@danielfireman A more idiomatic approach is always going to resonate with Go users for sure. I suspect your Go Fu is more powerful than mine. :) I will attempt to better acquaint myself with idiomatic coding styles in Go and also watch your approaches with it.

[danielfireman]

Thanks for the feedback!

I actually thought about this collision, but I ended concluding that it won't happen in most cases (at least in simple ones). This is because tableschema-go/csv could be used in many places where encoding/csv is being used. This is particularly true for simple cases (iterating and writing CSVs).

Do you have a case in mind that you would like to share? Or a missing feature?

[fils]

I'll see if I can think of one... I need to start trying some code and see. I'm wondering if there might be a case where CSV needs to be modified or manipulated prior to being used in tableschema?

also

Fils:fdgolang dfils$ ls
Gopkg.lock      Gopkg.toml      main.go         vendor
Fils:fdgolang dfils$ dep status
PROJECT                                     CONSTRAINT  VERSION  REVISION  LATEST   PKGS USED
github.com/frictionlessdata/tableschema-go  ^0.1.1      v0.1.1   c105f74   c105f74  3
github.com/satori/go.uuid                   *           v1.1.0   879c588   879c588  1
Fils:fdgolang dfils$ dep ensure
Fils:fdgolang dfils$ go build
# opencoredata.org/ocdGarden/frictionlessdata/fdgolang
./main.go:16: undefined: csv.FileSource
Fils:fdgolang dfils$

I'm trying to compile the example again.. see anything obvious I am missing?

[fils]

@danielfireman I am playing and learning... I think the example on the README.md needs to use FromFile not FromSource and the struct needs {} and tossing in a package main makes it run directly.

package main

import "github.com/frictionlessdata/tableschema-go/csv"

// struct representing each row of the table.
type person struct {
	Name string
	Age  uint16
}

func main() {
	t, _ := csv.New(csv.FromFile("data.csv"), csv.LoadHeaders()) // load table
	t.Infer()                                                    // infer the table schema
	t.Schema.SaveToFile("schema.json")                           // save inferred schema to file
	data := []person{}
	t.CastAll(&data) // casts the table data into the data slice.
}

can't tell you have thrilled I am to have this...
I have about 18,000 datasets I want to expose this way. Was using CSV for the Web, but happy to move to Frictionless and Go is my language of choice.

You going to do the datapackage package for Go next?

[danielfireman]

Thanks, @fils ! Sorry for the typos in the example. My bad. Fixed in commit 3543e54

I am super glad to read your words! Thanks a lot!

Yes, datapackage is my next goal.

About t.CastAll: it is worth to check t.Iter(), which allows efficient iteration without allocating the whole dataset in memory. That example would be:

package main

import "github.com/frictionlessdata/tableschema-go/csv"

// struct representing each row of the table.
type person struct {
	Name string
	Age  uint16
}

func main() {
    t, _ := csv.New(csv.FromFile("data.csv"), csv.LoadHeaders()) // load table
    t.Infer()                                                    // infer the table schema
    t.Schema.SaveToFile("schema.json")                           // save inferred schema to file
    var p person
    iter, _ := tab.Iter()
    for iter.Next() {
        iter.CastRow(&p)
    }
}

[danielfireman]

Oh .. forgot to comment on the namespace collision.

I'll see if I can think of one... I need to start trying some code and see. I'm wondering if there might be a case where CSV needs to be modified or manipulated prior to being used in tableschema?

That's a great point! My plan is to cover this the full need using tableschema-go. The writing needs are going to be fixed by #31 . Your considerations also allowed me to reconsider the Table API. I would say we should go with a more idiomatic approach for go: Readers and Writers.

What do you think?

[pwalsh]

Hi @fils . I'm the Product Owner on the Frictionless Data work we do at Open Knowledge International. I'd love to hear more about your 18,000 datasets, so I can potentially learn about new use cases and applications for the specs and tooling.

[danielfireman]

@fils Sure! I'll be rolling out those changes over this week and release a new version by Friday. I would like to start working on datapackages-go next week.

[danielfireman]

All the reading path is done. Closing this issue and following up with the writing path on issue #31

[danielfireman]

@fils Just released a new version with some API changes for reading (more idiomatic) and also adding writing capabilities.

Improved a lot the README.md

Please let me know if you have any problems or have suggestions.

Thanks!

[fils]

@danielfireman I'll pull down updates tomorrow and do some tests with reading and writing some example datasets with some of my own data. I'll link in my examples and experiences here.

[danielfireman]

Awesome! Looking forward to hearing from your experiences. Thanks! Em 10 de set de 2017 21:46, "Douglas Fils" <[email protected]> escreveu: @danielfireman <https://github.com/danielfireman> I'll pull down updates tomorrow and do some tests with reading and writing some example datasets with some of my own data. I'll link in my examples and experiences here. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#30 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIiWQ83rK9TRd9OME_B9rM1esT-CkYpSks5shILlgaJpZM4PJOsz> .