Comments (9)
This is (mostly) possible with a variadic template. First off, you must use
an array, since the size of a vector is not known at compile time. Second,
you need a helper template to take that array and "unroll" it into args.
It's a little tricky if you're new to TMP, but it's a common enough
construct.
Edit - I've pasted in some snippets of my working code. It's far from a MWE, but it will (hopefully) provide you with a starting place.
// http://stackoverflow.com/questions/16834851/passing-stdarray-as-arguments-of-template-variadic-function
// magic: generate integer sequence as function args, pass to variadic template
namespace indices_t {
template<int... Is>
struct seq { };
template<int N, int... Is>
struct gen_seq : gen_seq<N - 1, N - 1, Is...> { };
template<int... Is>
struct gen_seq<0, Is...> : seq<Is...> { };
}
// example calling function template
// the analogy to a "column name array" here is row_text
template<int... Is>
void _read_row(reader_t & in_csv, indices_t::seq<Is...>) {
// ...
while(
in_csv.read_row( row_text[Is]...)
) {
// do stuff
}
}
// call example -
// the effect is that _read_row is indices_t generates one arg, a numeric index, for each element of the array
_read_row(in_csv, indices_t::gen_seq<Ndat>());
hth,
-xian
from fast-cpp-csv-parser.
@helmingstay Thanks for your time Xian! This is a very interesting solution! However, wouldn't this require modifying the csv.h? In addition, I have two questions:
- I am the new function will only read the csv values as text (and does not deal with casting into types). Is this correct?
- Instead of this approach (which requires modifying csv.h), why not add one more signature like the following:
bool read_row(std::vector<std::string>& columns)
Would be interesting to see whether @ben-strasser has any plans of supporting this. If so, maybe @helmingstay can submit a pull request? I would highly prefer something that does not require me to maintain my own fork of the codebase.
from fast-cpp-csv-parser.
Just to reiterate, the code I added wasn't meant as a complete example, or as a solution to your particular issue. It was simply to A) let you know that your request was possible and B) point you in the right direction.
To answer your questions:
A. " However, wouldn't this require modifying the csv.h?"
I've simply added a few alias templates and helper classes that set policies, and mold the interface in csv.h to my liking. For example, I have a header with the following:
// convenience policy definition, trim spaces, comma sep, comments=#
template<size_t ncol>
using csv_t = io::CSVReader<ncol, io::trim_chars<' '>, io::no_quote_escape<','>, io::throw_on_overflow, io::single_line_comment<'#'> >;
- In my case, I'm reading everything as text and then manually parsing. As @ben-strasser notes in the README, "The builtin number parsers are pure convenience. If you need a slightly different syntax then use char* and do the parsing yourself."
- It is NOT POSSIBLE to determine the dimension of a vector at runtime. Thus, this code will not (and cannot be made to) compile. You could do this with a std::array containing char* (though be aware that, for the exact same reason, an array of std::strings is not possible).
I suggest spending a few days reading up on TMP and variadic templates in cpp, and then working through a few examples. Some of the questions that you're asking will be quickly clarified with some practice :)
from fast-cpp-csv-parser.
Is it possible to instead accept a vector containing columns with specific types to do this? Something like the following:
auto cols = new Columns{ ... three string cols ... }; _reader->read_row(cols);
will then populate cols[0], cols[1] and cols[2] as needed. Any suggestions on how this can be done?
You can do this:
vector<string>vec(3);
in.read_row(vec[0], vec[1], vec[2]);
This will get the data into a vector of strings without you having to declare a variable per column. Sounds to me like what you were asking for.
Instead of this approach (which requires modifying csv.h), why not add one more signature like the following:
bool read_row(std::vector<std::string>& columns)
CSVReader takes the number of columns as compile-time constant. A vector has a dynamic size. What should happen if the vector's size and the number of columns do not coincide? There are many options: assert fails, exception thrown, vector resized, segfault, stack trashing. No option is intuitive. On the other hand the simple
in.read_row(vec[0], vec[1], vec[2]);
makes it absolutely clear that it is the responsibility of the caller to make the vector the right size. Further, not having an additional method makes the interface thinner. Less code -> less bugs, Smaller interface -> easier to learn.
@helmingstay Be aware that the docs says to read as char*
and not as std::string
. From a performance point of view, there is a significant difference between those two.
from fast-cpp-csv-parser.
@ben-strasser "Be aware that the docs says to read as char* and not as std::string. From a performance point of view, there is a significant difference between those two."
Yes, thank you :)
In my case I actually do use std::string, mainly out of laziness, but I was careful to remove any mention of it in my examples (and explicitly point to your README comment).
from fast-cpp-csv-parser.
I respectfully think its quite useful to have dynamic fill for values in the in.read_row api. A common usecase is to process very different CSV files for analysis. How would one know in advance the number of columns and formulate that in the source code then? Is there another way to populate a variadic function?
from fast-cpp-csv-parser.
from fast-cpp-csv-parser.
The speed of this library is very good. The limitation of the static column typing I think can be solved. If you have a branch or commit I can test, I will be more than happy. For now, I will look at your proposal and sample code above and see if I can fix it.
Thanks for pointing out the difference between a variadic template and a variadic function!
from fast-cpp-csv-parser.
from fast-cpp-csv-parser.
Related Issues (20)
- How can i skip specified line? HOT 1
- Segmentation fault when running inside Boost Unit Test Framework HOT 2
- free(): invalid pointer HOT 4
- i18n README HOT 1
- My csv files can have changing number of columns
- Add ability to detect NULL values HOT 3
- Parse Single Line Without Loading File HOT 3
- Loss of precision on float reading HOT 4
- Indexing read_row HOT 1
- Count rows without processing them? HOT 1
- Possible to ignore columns in read_row()?
- Can the parsing performance be improved by using a precomputed index? HOT 2
- Progressbar support HOT 2
- Read file line by line
- Hope for more examples for code noobs
- Is it possible that I can wrap this library with lz4?
- Can I read the csv from console with this library?
- Usage of set_file_line is not clear HOT 1
- C4996 (function or variable may be unsafe) error for strncpy and fopen HOT 2
- Parsing a CSV with unknown number of columns HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fast-cpp-csv-parser.