Comments (5)
Related to this, I like the fact that we have a JsonValue that's useful on its own and then build a type provider on top of it to add type safety. This way we always have access to underlying JsonValue for edge cases when needed. The XmlProvider is similar, giving access to the underlying XElement.
I think we should have the same pattern for CsvProvider. I'm prototyping something that replaces the base classes CsvRow and CsvFile under RuntimeImplementation and promotes them to first class citizens, giving them more functionality, getting closer to the functionality of R's DataFrame (this includes dynamic lookup as described in #64). Then CsvProvider can built on top adding type safety, but you can always escape to the underlying values and do the .AsXxx like in the Json provider
from fsharp.data.
- I think keeping CSV in the name is probably a good idea (I expect that people know the name and realize that this is actually working for wider range of tabular data sources and I think nobody really expects that the comma in Comma-Separated-Values has to be a comma :-))
- I think we do not need multiple providers. The reason why this is needed for WorldBank is that there are default values for all parameters and so one version is not parameterized (
WorldBank.Countries....
). For CSV (etc.) we always need at least the input.
But:
-
I really like the idea of changing
CsvProvider
andCsvRow
to follow the same style asJsonValue
and be standalone types that people can use for dynamic access (I think we can pretty much follow the same pattern and have a module that adds dynamic operator and variousAsXxx
extensions).If you're happy to look into that, I'll leave it to you (if we do this, we'll need to add another
*.fsx
file with some documentation for the dynamic access).
from fsharp.data.
I've been doing a bunch of R code lately, so I'll try to convert some of it to use FSharp.Data instead so to get a feel what would work better as a JsonValue-like API
from fsharp.data.
With the latest changes from #122, we already have a decent enough dynamic API. I did a comparison between using the type provider, using the dynamic api, and using R here: https://gist.github.com/ovatsus/5354187
One advantage the dynamic version has is that we can slice the columns directly (https://gist.github.com/ovatsus/5354187#file-csvfile-fsx-L45), but we could eventually be able to do something like that with the typed version. On both cases, the average by column is not very easy to do, unless we consider a csv file to have similar operations to a matrix, and that's not easy to do in unless all the columns are of the same time
The R code is still more concise when doing filtering and mapping on the datasets, I think we have a lot of room of improvement here. FMat is able to get a Matlab/R-like syntax, maybe we could get some of that too. A possible idea would be something like this https://gist.github.com/ovatsus/5355630. I'm using the dynamic api and hardcoded a few things to make it look like the typed api. But even if we could make that work on the type provider version, I'm not very happy with it either. @tpetricek do you have any bright idea?
from fsharp.data.
I think the api is good enough for now, and the csv name is not ideal but it's ok, so I'm closing this. Let's keep things minimal until we have more real world feedback
from fsharp.data.
Related Issues (20)
- The design-time type 'FSharp.Data.Runtime.StructuralInference+InferenceMode' utilized by a type provider was not found HOT 8
- Support DateOnly and TimeOnly types HOT 2
- FSharp.Data.Core.Tests.CSharp.csproj: [NU1504] Duplicate 'PackageReference' items found. HOT 2
- s HOT 1
- JsonProvider - writable properties? HOT 1
- When closing VS 2022, the FSharp.Data.DesignTime stays on lock that prevents closing the editor. HOT 1
- Why are the core features (`CommonRuntime` folder) implemented in the Http project? HOT 6
- Question: an F# type provider backed by ML.NET for AI/learning HOT 1
- A way forward with the JsonProvider HOT 17
- Requesting a new beta package published to nuget.org HOT 3
- FSharp.Data.Runtime.Utilities is not published HOT 2
- Preserving linefeed style HOT 3
- A new FSharp.Data.JsonSaveOptions value needed HOT 3
- XmlProvider generates types based on name attribute in XSD schema rather than the type attribute HOT 5
- support empty space separator HOT 1
- update pictures in docs with #r nuget reference (no .dll ref)
- Update FSharp.Core from 5 to 6 or 7, to get less memory usage and better performance HOT 4
- System.Text.Json for JsonProvider? HOT 1
- Memory Limitations in XmlProvider HOT 10
- FS3033: Multiple definition of element error.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fsharp.data.