Giter Site home page Giter Site logo

vkhodygo / csv-parsers-comparison Goto Github PK

View Code? Open in Web Editor NEW

This project forked from univocity/csv-parsers-comparison

0.0 0.0 0.0 6.28 MB

Comparisons among all Java-based CSV parsers in existence (not really)

License: Apache License 2.0

Java 98.23% Shell 1.77%
java benchmarks csv-parser

csv-parsers-comparison's Introduction

Fork of uniVocity/csv-parsers-comparison/

Summary

Currently, we are only testing parsing performance. As the input file, we will be using the famous GeoLite2-City-Blocks-IPv4.csv, which is made available for free by Maxmind. It contains more than 3 million rows, which should be sufficient for our test. SimpleFlatMapper and Jackson are the fastest one in those runs. SFM tends to be faster in most cases apart from the 1000 rows with no quotes in java 6 and 7.

Building and Running

Univocity has a stiff initialisation price to pay that makes a lean heavily on the run under 100000 rows.

Prerequisites: Git, Apache Maven 3 and Java 1.8+. Some parser can have sensibly different resuslt depending on the java version. TieredCompilation - which is enabled by default in java 8 - seems to have mixed results accross the parsers. Jcsv for example is pretty bad under java 6, pretty good on java 7 but degrade quote a bit in java 8.

If you wish to reproduce our performance results:

$ git clone https://github.com/azazar/csv-parsers-comparison.git
$ cd csv-parsers-comparison
$ mvn package exec:java

Our test is very simple and involves just counting the number of rows read from the input file. The implementation using each parser is here.

Because I get the same result running the jmh and the original benchmark running on my machine I doubt the code is the issue.

Important

The input file is not RFC 4180 compliant. We generate a compliant version using the HugeFileGenerator class to test the parsers against a generated file with the same data, but enclosed within quotes and properly escaped.

It's important to notice that there's no such thing as a CSV standard and we do not recommend you to use parsers that follow the RFC strictly, as they will blow up in face of non-compliant inputs. The reality is: your parser must be ready to process crooked data instead of going belly up. In the end, your client is the one who tells you what you must swallow, and in many circumstances it's not up to you to decide how your data is going to be generated.

We generate a RFC compliant version to give those sensitive parsers a chance to see how they perform. Once again, we consider their usage risky.

BeanIO and GenJava have the worst performance overall.

This is the list of all parsers currently tested.

Parser Version Website
uniVocity-parsers' CsvParser 2.8.1 www.univocity.com
CSVeed 0.6.0 csveed.org
Apache Commons CSV 1.6 [commons.apache.org/proper/commons-csv] (http://commons.apache.org/proper/commons-csv)
OpenCSV 4.5 opencsv.sourceforge.net
SuperCSV 2.4.0 supercsv.sourceforge.net
JavaCSV 2.0 sourceforge.net/projects/javacsv
jCSV 1.4.0 code.google.com/p/jcsv
flatpack 4.0.4 flatpack.sourceforge.net
SimpleCSV 2.1 github.com/quux00/simplecsv
gj-csv 1.0 ?
esperio-csv 8.1.0 www.espertech.com
way-io 2.1.0 www.objectos.com.br
beanIO 2.1.0 beanio.org
jackson-dataformat-csv 2.9.8 github.com/FasterXML/jackson-dataformat-csv
OsterMiller Utils 1.07.00 ostermiller.org/utils/CSV.html
SimpleFlatMapper CSV parser 6.4.0 github.com/arnaudroger/SimpleFlatMapper
Diergo Easy CSV Streamable github.com/aburmeis/decs
Product Collections 1.4.5 github.com/marklister/product-collections
CSV-Simple 1.1.2 https://github.com/azazar/csv-simple
Sesseltjonna-csv 1.0.11 github.com/skjolber/sesseltjonna-csv

Benchmarks

Parser Version Website
uniVocity-parsers' CsvParser 1.3.0 www.univocity.com
CSVeed 0.4.0 csveed.org
Apache Commons CSV 1.0 [commons.apache.org/proper/commons-csv] (http://commons.apache.org/proper/commons-csv)
OpenCSV 2.3 opencsv.sourceforge.net
SuperCSV 2.2.0 supercsv.sourceforge.net
JavaCSV 2.0 sourceforge.net/projects/javacsv
jCSV 1.4.0 code.google.com/p/jcsv
flatpack 3.4.2 flatpack.sourceforge.net
SimpleCSV 2.0 github.com/quux00/simplecsv
gj-csv 1.0 ?
esperio-csv 4.11.0 www.espertech.com
way-io 1.6.0 www.objectos.com.br
beanIO 2.1.0 beanio.org
DataPipeline's CSVReader* 2.3.5 northconcepts.com/data-pipeline/
jackson-dataformat-csv 2.4.2 github.com/FasterXML/jackson-dataformat-csv
OsterMiller Utils 1.0.6 ostermiller.org/utils/CSV.html
SimpleFlatMapper CSV parser 1.0.0rc2 github.com/arnaudroger/SimpleFlatMapper

worldcitiespop.txt

Statistics (updated 18th of March, 2019)

echo "performance" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor;
echo "2000000" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_min_freq;
echo "2000000" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_max_freq
## Statistics (updated 28th of November, 2014)

Results will vary depending on your setup and hardware, here is mine: 

java -jar target/benchmarks.jar -tu us \
  -wi 10 -i 10 -f 5 \
  -p nbRows=1,10,100,1000,10000,100000,1000000,-1 \
  -rf csv -foe -p inputFile=./worldcitiespop.txt \
  -rff jmh.csv -jvmArgs="-Xmx1g -Xms1g"

Results will vary depending on your setup and hardware. For reference, here's my (very) modest hardware, an ultrabook:

  • CPU: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz
  • RAM: 24 GB
  • Storage: 500GB RAID1 SSD
  • OS: Fedora release 29 64-bit
  • JDK: OpenJDK Runtime Environment (build 1.8.0_201-b09) 64-Bit Server VM
  • JDK: OpenJDK Runtime Environment 18.9 (build 11.0.1+13) openjdk version "11.0.1"
  • CPU: Intel i5-3337U @ 1.8 GHz
  • RAM: 4 GB
  • Storage: 128GB SSD drive
  • OS: Arch Linux 64-bit
  • JDK: 1.8.0_25 64-bit (Linux)
  • JDK: 1.7.0_71 64-bit (Linux)
  • JDK: 1.6.0_45 64-bit (Linux)
  • Intel(R) Core(TM) i5 CPU 750 @ 2.67GHz
  • Oracle Java 1.6.0_45
  • Oracle Java 1.7.0_72
  • Oracle Java 1.8.0_25
  • 6G Ram
  • Ubuntu
  • HDD

Note uniVocity-parsers provides an option to select the fields you are interested in, and our parsers will execute faster by not processing values that are not selected. It makes quite a difference in performance but we removed this test as the other parsers don't have a similar feature. Excel data Note uniVocity-parsers provides an option to select the fields you are interested in, and our parsers will execute faster by not processing values that are not selected. It makes quite a difference in performance but we removed this test as the other parsers don't have a similar feature.

Processing 3,230,544 rows of non RFC 4180 compliant input. No quoted values.

Java 6 Results

No quote results

Processing 3,173,958 rows of non RFC 4180 compliant input.

the unit is us per reading the file of nbrows. The lower the better.

JDK 8

Parser Average time % Slower than best Best time Worst time
uniVocity CSV parser 1314 ms Best time! 1308 ms 1334 ms
SimpleFlatMapper CSV parser 1352 ms 2% 1342 ms 1394 ms
Gen-Java CSV 2098 ms 59% 2090 ms 2124 ms
Jackson CSV parser 2206 ms 67% 2204 ms 2214 ms
JCSV Parser 2228 ms 69% 2227 ms 2231 ms
Java CSV Parser 2381 ms 81% 2370 ms 2411 ms
SuperCSV 2387 ms 81% 2377 ms 2414 ms
Product Collections parser 2391 ms 81% 2379 ms 2403 ms
Simple CSV parser 2437 ms 85% 2426 ms 2473 ms
Oster Miller CSV parser 2654 ms 101% 2649 ms 2663 ms
CSV-Simple 2672 ms 103% 2661 ms 2696 ms
OpenCSV 2957 ms 125% 2950 ms 2961 ms
Way IO Parser 3774 ms 187% 3767 ms 3782 ms
Apache Commons CSV 3853 ms 193% 3834 ms 3905 ms
Bean IO Parser 4567 ms 247% 4544 ms 4629 ms
Esperio CSV parser 23906 ms 1719% 23842 ms 24007 ms

JDK 11

Parser/nbrows 1 10 100 1000 10000 100000 1000000 3173959
JacksonParser 2.4.2 20.56 28.91 109.42 959.92 9,259.31 91,718.92 904,262.33 2,862,200.83
SimpleFlatMapperParser 1.2.0 20.97 27.70 92.92 890.45 7,425.59 75,174.15 736,202.60 2,342,475.25
JCsvParser 1.4.0 22.00 34.37 162.13 1,341.85 12,723.23 127,770.35 1,246,264.46 4,522,937.51
SimpleCsvParser 2.0 22.39 36.34 177.18 1,387.69 13,706.27 167,896.96 1,641,417.61 5,221,632.46
OpenCsvParser 2.3 21.65 33.82 149.24 1,396.74 12,959.22 133,597.68 1,304,542.01 4,156,496.28
SuperCsvParser 2.2.0 22.46 35.18 169.48 1,430.02 13,860.77 165,970.79 1,606,424.38 5,256,495.20
OsterMillerParser 1.07.00 23.61 35.75 157.77 1,555.74 14,918.44 151,041.99 1,484,759.67 4,746,388.14
JavaCsvParser 2.0 31.09 41.39 176.16 1,588.15 15,130.67 152,590.81 1,504,618.33 4,806,283.53
WayIoParser 1.6.0 28.39 44.13 201.51 1,836.43 17,624.63 174,649.84 1,764,296.31 5,702,440.66
CommonsCsvParser 1.0 20.18 35.31 187.04 1,870.36 17,812.89 179,474.87 1,747,955.82 5,539,298.62
GenJavaParser 1.0 22.14 44.03 310.80 3,645.57 34,678.04 346,876.24 3,558,724.86 9,636,547.34
UnivocityParser 1.0.3 4,447.20 4,457.70 4,568.01 5,729.47 17,630.42 132,168.34 1,219,204.75 3,909,024.04

JDK 7

Parser Average time % Slower than best Best time Worst time
SimpleFlatMapper CSV parser 1375 ms Best time! 1366 ms 1382 ms
uniVocity CSV parser 1428 ms 3% 1356 ms 1489 ms
Jackson CSV parser 2082 ms 51% 2043 ms 2132 ms
Product Collections parser 2124 ms 54% 2112 ms 2144 ms
JCSV Parser 2193 ms 59% 2188 ms 2205 ms
Java CSV Parser 2206 ms 60% 2193 ms 2220 ms
Oster Miller CSV parser 2545 ms 85% 2510 ms 2648 ms
CSV-Simple 2695 ms 96% 2679 ms 2709 ms
OpenCSV 2804 ms 103% 2782 ms 2843 ms
Simple CSV parser 2868 ms 108% 2819 ms 2983 ms
SuperCSV 2919 ms 112% 2906 ms 2959 ms
Apache Commons CSV 3596 ms 161% 3580 ms 3635 ms
Way IO Parser 3693 ms 168% 3675 ms 3707 ms
Gen-Java CSV 3813 ms 177% 3802 ms 3819 ms
Bean IO Parser 4955 ms 260% 4869 ms 5018 ms
Esperio CSV parser 22837 ms 1560% 22767 ms 22907 ms
  • CSVeed was unable to process the file and threw exception with the message Illegal state transition: Parsing symbol QUOTE_SYMBOL [34] in state INSIDE_FIELD. Latency difference in % from jackson parser Difference from jackson parser

JDK 6

Parser Average time % Slower than best Best time Worst time
SimpleFlatMapper CSV parser 1749 ms Best time! 1718 ms 1790 ms
uniVocity CSV parser 1911 ms 9% 1796 ms 2031 ms
Jackson CSV parser 1983 ms 13% 1906 ms 2103 ms
OpenCSV 2611 ms 49% 2567 ms 2675 ms
JCSV Parser 2653 ms 51% 2600 ms 2707 ms
Java CSV Parser 2782 ms 59% 2706 ms 2879 ms
Data pipeline 2805 ms 60% 2688 ms 2894 ms
Simple CSV parser 3098 ms 77% 2945 ms 3421 ms
Oster Miller CSV parser 3122 ms 78% 3022 ms 3210 ms
SuperCSV 3185 ms 82% 3059 ms 3351 ms
Apache Commons CSV 3358 ms 91% 3280 ms 3445 ms
Way IO Parser 3743 ms 114% 3683 ms 3827 ms
Gen-Java CSV 7144 ms 308% 6991 ms 7278 ms
  • Esperio-csv and CSVeed were unable to process the file and threw exceptions.
  • Flatpack hanged so I had to remove it from the test here.
  • BeanIO threw an exception I could understand and debug. Turns out it is unable to parse fields when the quote character is part of the value, e.g. value1, val"ue2, value3 .

Processing 3,173,958 rows of RFC 4180 compliant input.

Quote results

JDK 8

Parser Average time % Slower than best Best time Worst time
uniVocity CSV parser 2270 ms Best time! 2178 ms 2352 ms
Jackson CSV parser 2416 ms 6% 2294 ms 2490 ms
SuperCSV 3116 ms 37% 2956 ms 3267 ms
JCSV Parser 3172 ms 39% 3049 ms 3325 ms
SimpleFlatMapper CSV parser 3405 ms 50% 3177 ms 3709 ms
Data pipeline 3424 ms 50% 3355 ms 3478 ms
Apache Commons CSV 3620 ms 59% 3500 ms 3769 ms
Simple CSV parser 3793 ms 67% 3635 ms 3896 ms
Java CSV Parser 3864 ms 70% 3703 ms 4046 ms
OpenCSV 4115 ms 81% 3955 ms 4532 ms
Oster Miller CSV parser 4165 ms 83% 3988 ms 4343 ms
Esperio CSV parser 5355 ms 135% 5122 ms 5591 ms
Way IO Parser 5400 ms 137% 5016 ms 5588 ms
Gen-Java CSV 8558 ms 277% 8106 ms 8844 ms
Bean IO Parser 10796 ms 375% 10504 ms 11341 ms

the unit is us per reading the file of nbrows. The lower the better.

JDK 7

Parser Average time % Slower than best Best time Worst time
uniVocity CSV parser 2352 ms Best time! 2286 ms 2449 ms
Jackson CSV parser 2432 ms 3% 2234 ms 2627 ms
SimpleFlatMapper CSV parser 2556 ms 8% 2373 ms 2836 ms
Data pipeline 3281 ms 39% 3084 ms 3596 ms
SuperCSV 3296 ms 40% 2989 ms 3500 ms
JCSV Parser 3305 ms 40% 3145 ms 3692 ms
OpenCSV 3859 ms 64% 3671 ms 4092 ms
Java CSV Parser 3884 ms 65% 3749 ms 4139 ms
Oster Miller CSV parser 4054 ms 72% 3804 ms 4255 ms
Simple CSV parser 4246 ms 80% 4093 ms 4516 ms
Apache Commons CSV 4303 ms 82% 4197 ms 4405 ms
Way IO Parser 5095 ms 116% 4935 ms 5270 ms
Esperio CSV parser 6001 ms 155% 5760 ms 6375 ms
Gen-Java CSV 10918 ms 364% 10345 ms 11305 ms
Bean IO Parser 12839 ms 445% 11992 ms 13641 ms

JDK 6

Parser Average time % Slower than best Best time Worst time
uniVocity CSV parser 2395 ms Best time! 2368 ms 2444 ms
SimpleFlatMapper CSV parser 2539 ms 6% 2480 ms 2582 ms
Jackson CSV parser 2809 ms 17% 2669 ms 2982 ms
Data pipeline 3197 ms 33% 3067 ms 3395 ms
JCSV Parser 3423 ms 42% 3293 ms 3678 ms
OpenCSV 3570 ms 49% 3457 ms 3847 ms
Java CSV Parser 3745 ms 56% 3541 ms 3977 ms
SuperCSV 3808 ms 58% 3638 ms 4062 ms
Oster Miller CSV parser 3944 ms 64% 3750 ms 4161 ms
Simple CSV parser 3962 ms 65% 3776 ms 4226 ms
Apache Commons CSV 4315 ms 80% 4189 ms 4448 ms
Way IO Parser 4667 ms 94% 4566 ms 4773 ms
Esperio CSV parser 5625 ms 134% 5496 ms 5778 ms
Gen-Java CSV 8977 ms 274% 8759 ms 9293 ms
Bean IO Parser 10848 ms 352% 10385 ms 11225 ms
  • CSVeed was unable to process the file and threw exception with the message "Parsing symbol OTHER_SYMBOL [44] in state ESCAPING".
  • Flatpack blew up the Java heap space so I had to remove it from the test here.
Parser/nbrows 1 10 100 1000 10000 100000 1000000 3173959
JacksonParser 2.4.2 20.80 31.80 144.23 1,462.09 13,966.33 140,360.80 1,380,760.39 4,401,189.30
SimpleFlatMapperParser 1.2.0 21.08 31.96 104.00 1,027.17 12,013.12 100,785.66 977,777.05 2,647,928.02
JCsvParser 1.4.0 22.94 41.51 228.19 2,156.42 21,638.66 221,701.13 2,117,656.18 6,822,423.28
SimpleCsvParser 2.0 22.51 40.04 217.32 1,787.30 19,384.62 200,927.10 1,965,590.72 6,284,008.83
OpenCsvParser 2.3 22.75 41.31 231.37 2,020.68 18,658.59 189,861.09 1,857,790.99 5,969,062.93
SuperCsvParser 2.2.0 23.16 42.58 233.84 1,996.42 19,450.24 200,009.79 1,968,554.76 6,261,176.90
OsterMillerParser 1.07.00 23.95 41.54 212.57 1,796.83 18,045.74 183,739.90 1,801,485.56 5,831,664.22
JavaCsvParser 2.0 31.81 43.27 194.01 1,820.18 17,776.71 184,548.24 1,769,503.49 5,613,036.50
WayIoParser 1.6.0 29.39 52.39 250.10 2,522.89 23,610.58 234,847.74 2,293,163.63 7,435,385.66
CommonsCsvParser 1.0 21.12 42.51 254.67 2,558.45 24,915.98 247,869.21 2,427,771.47 7,717,016.95
GenJavaParser 1.0 22.90 51.06 388.23 4,516.87 43,293.45 439,501.07 4,696,391.12 14,800,000.00
UnivocityParser 1.0.3 4,467.26 4,476.80 4,603.88 5,994.07 20,270.97 155,576.21 1,459,677.87 4,639,997.25
Bean IO 2.1.0 24.12 61.14 501.49 5,656.26 54,608.29 587,428.25 5,652,539.16 17,933,796.24

Reliability (updated 9th of October, 2017)

Latency difference in % from jackson parser Difference from jackson parser

The following parsers were unable to process the RFC 4180 compliant file correctness.csv. This test is executed using the class CorrectnessComparison.java

Java 7 Results

No quote results

The exact same errors have been reported 3 years ago when I last updated this page. Just avoid these parsers.

Conclusion

the unit is us per reading the file of nbrows. The lower the better.

Parser/nbrows 1 10 100 1000 10000 100000 1000000 3173959
JacksonParser 2.4.2 20.24 28.21 111.92 955.18 9,493.98 94,809.19 930,487.07 2,976,824.49
SimpleFlatMapperParser 1.2.0 20.55 27.28 91.43 1,023.67 7,459.68 74,660.33 744,521.08 2,321,910.07
JCsvParser 1.4.0 21.26 28.68 105.15 1,011.24 9,524.07 94,574.76 934,419.76 2,811,022.42
SimpleCsvParser 2.0 22.04 34.95 172.58 1,412.42 16,995.40 160,371.04 1,603,714.95 4,987,105.28
OpenCsvParser 2.3 21.57 33.50 165.77 1,627.72 14,437.39 148,230.18 1,424,704.26 4,355,408.10
SuperCsvParser 2.2.0 22.53 34.14 146.19 1,641.76 15,492.30 133,140.56 1,282,506.94 3,968,221.65
OsterMillerParser 1.07.00 23.32 32.93 142.80 1,254.54 11,989.83 122,604.50 1,205,519.51 4,075,179.72
JavaCsvParser 2.0 29.17 40.69 171.89 1,583.93 15,452.34 151,460.41 1,497,675.96 4,789,507.69
WayIoParser 1.6.0 28.63 44.43 212.83 1,896.42 19,045.36 181,337.58 1,832,200.21 5,644,370.67
CommonsCsvParser 1.0 19.28 34.18 178.34 1,780.89 16,980.91 171,601.39 1,667,304.99 5,343,597.19
GenJavaParser 1.0 22.18 43.27 308.31 3,376.42 38,819.72 391,391.19 4,099,708.92 9,630,511.79
UnivocityParser 1.0.3 4,329.47 4,332.47 4,516.22 5,855.99 17,534.36 130,439.24 1,217,036.03 3,875,463.28
Currently, three parsers stand out as the fastest CSV parsers for Java:

uniVocity-parsers, SimpleFlatMapper and Jackson CSV. Keep in mind that Simpleflatmapper is a very simple implementation that does not provide any customization options. Results are affected by a simple change in the JDK version, however these three parsers are always at the top. Latency difference in % from jackson parser Difference from jackson parser uniVocity-parsers, SimpleFlatMapper and Jackson CSV. Keep in mind that Simpleflatmapper is a very simple implementation that does not provide any customization options.

We will keep working to improve the performance of our parsers, and will try to update the results of this benchmark every time a new parser is added to the list.

Head on to the uniVocity-parsers github page to get access to its source code and documentation. Contributions are welcome.

Quote results

the unit is us per reading the file of nbrows. The lower the better.

Parser/nbrows 1 10 100 1000 10000 100000 1000000 3173959
JacksonParser 2.4.2 20.47 31.27 148.96 1,453.41 13,931.26 138,685.49 1,365,335.11 4,363,888.96
SimpleFlatMapperParser 1.2.0 20.80 32.08 107.77 942.46 11,647.54 102,628.48 982,690.25 2,635,456.90
JCsvParser 1.4.0 21.80 33.19 146.64 1,360.52 13,421.54 136,428.87 1,325,533.89 4,236,211.65
SimpleCsvParser 2.0 22.73 40.94 212.33 2,203.63 20,136.39 203,944.21 2,002,961.29 6,348,187.18
OpenCsvParser 2.3 22.13 36.84 177.19 2,185.44 19,897.37 199,481.46 2,081,256.96 6,610,777.70
SuperCsvParser 2.2.0 22.57 36.87 168.28 1,714.92 15,760.05 163,915.88 1,616,134.18 4,932,034.29
OsterMillerParser 1.07.00 23.53 39.27 195.17 1,706.05 17,354.48 168,445.49 1,673,356.38 5,297,194.44
JavaCsvParser 2.0 29.40 43.03 193.09 1,840.75 17,614.13 176,080.17 1,732,943.33 5,525,471.43
WayIoParser 1.6.0 28.73 49.97 230.92 2,637.67 24,563.24 246,650.74 2,451,107.83 7,725,632.94
CommonsCsvParser 1.0 20.36 41.24 252.90 2,385.53 24,277.71 244,260.14 2,403,832.58 7,658,031.91
GenJavaParser 1.0 22.58 50.01 378.46 4,069.74 48,406.71 479,780.80 4,869,382.85 15,486,546.01
UnivocityParser 1.0.3 4,349.93 4,356.36 4,492.51 6,085.43 20,121.76 154,325.93 1,460,668.11 4,618,588.49
Bean IO 2.1.0 24.15 62.62 503.72 5,371.11 59,605.92 608,495.75 5,708,588.10 18,253,867.13

Latency difference in % from jackson parser Difference from jackson parser

Java 8 Results

No quote results

the unit is us per reading the file of nbrows. The lower the better.

Parser/nbrows 1 10 100 1000 10000 100000 1000000 3173959
JacksonParser 2.4.2 16.97 24.77 100.55 901.69 8,896.07 94,738.75 853,498.25 2,632,276.44
SimpleFlatMapperParser 1.2.0 17.53 23.58 94.14 885.84 8,468.54 71,266.72 687,330.28 2,544,713.83
JCsvParser 1.4.0 18.08 27.02 117.38 1,156.94 10,631.32 108,310.56 1,043,085.15 3,538,204.72
SimpleCsvParser 2.0 19.18 32.37 169.22 1,628.87 15,464.43 157,553.29 1,562,049.57 4,877,498.34
OpenCsvParser 2.3 18.84 31.93 161.40 1,554.06 15,182.50 151,518.48 1,518,520.20 4,789,898.43
SuperCsvParser 2.2.0 19.21 32.62 168.36 1,302.04 12,223.01 128,351.83 1,254,158.67 4,406,608.49
OsterMillerParser 1.07.00 20.10 30.42 138.27 1,240.67 12,019.66 122,622.17 1,215,447.04 3,778,397.37
JavaCsvParser 2.0 26.02 37.50 167.64 1,540.67 14,925.34 154,624.11 1,513,953.07 4,714,272.93
WayIoParser 1.6.0 24.68 41.87 217.66 2,065.52 19,179.45 193,070.96 1,924,073.63 6,076,373.58
CommonsCsvParser 1.0 16.41 30.18 169.53 1,728.94 15,484.71 170,333.01 1,683,754.64 5,322,571.13
GenJavaParser 1.0 18.45 42.60 308.68 3,326.24 32,579.51 339,035.43 3,756,598.00 8,704,666.27
UnivocityParser 1.0.3 4,201.40 4,218.65 4,336.74 5,640.11 17,485.71 131,026.03 1,219,344.32 3,840,187.86

Latency difference in % from jackson parser Difference from jackson parser

Quote results

the unit is us per reading the file of nbrows. The lower the better.

Parser/nbrows 1 10 100 1000 10000 100000 1000000 3173959
JacksonParser 2.4.2 17.05 27.53 133.20 1,363.76 13,582.91 141,354.32 1,198,963.25 3,855,814.01
SimpleFlatMapperParser 1.2.0 17.96 28.95 103.55 903.07 10,399.61 98,014.00 987,683.82 2,579,806.64
JCsvParser 1.4.0 18.35 29.79 144.35 1,392.68 13,206.78 140,968.94 1,360,875.42 4,232,535.86
SimpleCsvParser 2.0 19.60 35.99 205.69 2,017.18 19,509.33 195,672.45 1,923,729.58 6,147,489.91
OpenCsvParser 2.3 19.75 38.65 225.62 2,272.33 20,009.25 204,661.36 2,046,529.59 6,358,606.13
SuperCsvParser 2.2.0 19.59 34.89 186.03 1,664.23 16,066.05 164,121.47 1,609,226.70 5,159,231.48
OsterMillerParser 1.07.00 20.78 35.23 181.34 1,656.05 16,340.60 167,926.99 1,666,674.45 5,311,749.90
JavaCsvParser 2.0 26.44 39.48 186.03 1,814.79 17,795.90 181,730.28 1,796,715.00 5,736,254.13
WayIoParser 1.6.0 25.16 49.25 282.85 2,645.04 24,383.32 249,550.40 2,392,075.74 7,806,415.77
CommonsCsvParser 1.0 16.87 34.23 209.50 2,074.49 20,196.62 200,649.14 1,974,438.21 6,298,800.79
GenJavaParser 1.0 19.01 47.86 373.35 3,941.18 39,546.31 422,574.06 4,899,920.83 16,005,336.01
UnivocityParser 1.0.3 4,208.47 4,205.89 4,359.03 5,949.49 20,120.54 153,626.12 1,445,956.53 4,607,323.12
Bean IO 2.1.0 20.41 58.87 463.19 4,987.58 52,391.02 535,415.79 6,081,133.21 19,282,239.96

Latency difference in % from jackson parser Difference from jackson parser

Comparaison across Java 6, 7 and 8

the unit is us per reading the file of 3173959 rows. The lower the better.

Parser Java 6 Java 7 Java 8
SimpleFlatMapperParser 2,342,475.25 2,321,910.07 2,544,713.83
JCsvParser 4,522,937.51 2,811,022.42 3,538,204.72
JacksonParser 2,862,200.83 2,976,824.49 2,632,276.44
UnivocityParser 3,909,024.04 3,875,463.28 3,840,187.86
SuperCsvParser 5,256,495.20 3,968,221.65 4,406,608.49
OsterMillerParser 4,746,388.14 4,075,179.72 3,778,397.37
OpenCsvParser 4,156,496.28 4,355,408.10 4,789,898.43

Effect of java version

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.