pyexcel/pyexcel-ods

It is a plugin to pyexcel and provides the capbility to read, manipulate and write data in ods formats using odfpy.

pyexcel - Let you focus on data, instead of file formats

Support the project

If your company has embedded pyexcel and its components into a revenue generating product, please support me on github, patreon or bounty source to maintain the project and develop it further.

If you are an individual, you are welcome to support me too and for however long you feel like. As my backer, you will receive early access to pyexcel related contents.

And your issues will get prioritized if you would like to become my patreon as pyexcel pro user.

With your financial support, I will be able to invest a little bit more time in coding, documentation and writing interesting posts.

Known constraints

Fonts, colors and charts are not supported.

Nor to read password protected xls, xlsx and ods files.

Introduction

Feature Highlights

A list of supported file formats

file format	definition
csv	comma separated values
tsv	tab separated values
csvz	a zip file that contains one or many csv files
tsvz	a zip file that contains one or many tsv files
xls	a spreadsheet file format created by MS-Excel 97-2003
xlsx	MS-Excel Extensions to the Office Open XML SpreadsheetML File Format.
xlsm	an MS-Excel Macro-Enabled Workbook file
ods	open document spreadsheet
fods	flat open document spreadsheet
json	java script object notation
html	html table of the data structure
simple	simple presentation
rst	rStructured Text presentation of the data
mediawiki	media wiki table

1. One application programming interface(API) to handle multiple data sources:
   • physical file
   • memory file
   • SQLAlchemy table
   • Django Model
   • Python data structures: dictionary, records and array
2. One API to read and write data in various excel file formats.
3. For large data sets, data streaming are supported. A genenerator can be returned to you. Checkout iget_records, iget_array, isave_as and isave_book_as.

Installation

You can install pyexcel via pip:

$ pip install pyexcel

or clone it and install it:

$ git clone https://github.com/pyexcel/pyexcel.git
$ cd pyexcel
$ python setup.py install

One liners

This section shows you how to get data from your excel files and how to export data to excel files in one line

Read from the excel files

Get a list of dictionaries

Suppose you want to process History of Classical Music:

History of Classical Music:

Name	Period	Representative Composers
Medieval	c.1150-c.1400	Machaut, Landini
Renaissance	c.1400-c.1600	Gibbons, Frescobaldi
Baroque	c.1600-c.1750	JS Bach, Vivaldi
Classical	c.1750-c.1830	Joseph Haydn, Wolfgan Amadeus Mozart
Early Romantic	c.1830-c.1860	Chopin, Mendelssohn, Schumann, Liszt
Late Romantic	c.1860-c.1920	Wagner,Verdi
Modernist	20th century	Sergei Rachmaninoff,Calude Debussy

Let's get a list of dictionary out from the xls file:

>>> records = p.get_records(file_name="your_file.xls")

And let's check what do we have:

>>> for row in records:
...     print(f"{row['Representative Composers']} are from {row['Name']} period ({row['Period']})")
Machaut, Landini are from Medieval period (c.1150-c.1400)
Gibbons, Frescobaldi are from Renaissance period (c.1400-c.1600)
JS Bach, Vivaldi are from Baroque period (c.1600-c.1750)
Joseph Haydn, Wolfgan Amadeus Mozart are from Classical period (c.1750-c.1830)
Chopin, Mendelssohn, Schumann, Liszt are from Early Romantic period (c.1830-c.1860)
Wagner,Verdi are from Late Romantic period (c.1860-c.1920)
Sergei Rachmaninoff,Calude Debussy are from Modernist period (20th century)

Get two dimensional array

Instead, what if you have to use pyexcel.get_array to do the same:

>>> for row in p.get_array(file_name="your_file.xls", start_row=1):
...     print(f"{row[2]} are from {row[0]} period ({row[1]})")
Machaut, Landini are from Medieval period (c.1150-c.1400)
Gibbons, Frescobaldi are from Renaissance period (c.1400-c.1600)
JS Bach, Vivaldi are from Baroque period (c.1600-c.1750)
Joseph Haydn, Wolfgan Amadeus Mozart are from Classical period (c.1750-c.1830)
Chopin, Mendelssohn, Schumann, Liszt are from Early Romantic period (c.1830-c.1860)
Wagner,Verdi are from Late Romantic period (c.1860-c.1920)
Sergei Rachmaninoff,Calude Debussy are from Modernist period (20th century)

where start_row skips the header row.

Get a dictionary

You can get a dictionary too:

>>> my_dict = p.get_dict(file_name="your_file.xls", name_columns_by_row=0)

And let's have a look inside:

>>> from pyexcel._compact import OrderedDict
>>> isinstance(my_dict, OrderedDict)
True
>>> for key, values in my_dict.items():
...     print(key + " : " + ','.join([str(item) for item in values]))
Name : Medieval,Renaissance,Baroque,Classical,Early Romantic,Late Romantic,Modernist
Period : c.1150-c.1400,c.1400-c.1600,c.1600-c.1750,c.1750-c.1830,c.1830-c.1860,c.1860-c.1920,20th century
Representative Composers : Machaut, Landini,Gibbons, Frescobaldi,JS Bach, Vivaldi,Joseph Haydn, Wolfgan Amadeus Mozart,Chopin, Mendelssohn, Schumann, Liszt,Wagner,Verdi,Sergei Rachmaninoff,Calude Debussy

Please note that my_dict is an OrderedDict.

Get a dictionary of two dimensional array

Suppose you have a multiple sheet book as the following:

Top Violinist:

Name	Period	Nationality
Antonio Vivaldi	1678-1741	Italian
Niccolo Paganini	1782-1840	Italian
Pablo de Sarasate	1852-1904	Spainish
Eugene Ysaye	1858-1931	Belgian
Fritz Kreisler	1875-1962	Astria-American
Jascha Heifetz	1901-1987	Russian-American
David Oistrakh	1908-1974	Russian
Yehundi Menuhin	1916-1999	American
Itzhak Perlman	1945-	Israeli-American
Hilary Hahn	1979-	American

Noteable Violin Makers:

Maker	Period	Country
Antonio Stradivari	1644-1737	Cremona, Italy
Giovanni Paolo Maggini	1580-1630	Botticino, Italy
Amati Family	1500-1740	Cremona, Italy
Guarneri Family	1626-1744	Cremona, Italy
Rugeri Family	1628-1719	Cremona, Italy
Carlo Bergonzi	1683-1747	Cremona, Italy
Jacob Stainer	1617-1683	Austria

Most Expensive Violins:

Name	Estimated Value	Location
Messiah Stradivarious	$ 20,000,000	Ashmolean Museum in Oxford, England
Vieuxtemps Guarneri	$ 16,000,000	On loan to Anne Akiko Meyers
Lady Blunt	$ 15,900,000	Anonymous bidder

Here is the code to obtain those sheets as a single dictionary:

>>> book_dict = p.get_book_dict(file_name="book.xls")

And check:

>>> isinstance(book_dict, OrderedDict)
True
>>> import json
>>> for key, item in book_dict.items():
...     print(json.dumps({key: item}))
{"Most Expensive Violins": [["Name", "Estimated Value", "Location"], ["Messiah Stradivarious", "$ 20,000,000", "Ashmolean Museum in Oxford, England"], ["Vieuxtemps Guarneri", "$ 16,000,000", "On loan to Anne Akiko Meyers"], ["Lady Blunt", "$ 15,900,000", "Anonymous bidder"]]}
{"Noteable Violin Makers": [["Maker", "Period", "Country"], ["Antonio Stradivari", "1644-1737", "Cremona, Italy"], ["Giovanni Paolo Maggini", "1580-1630", "Botticino, Italy"], ["Amati Family", "1500-1740", "Cremona, Italy"], ["Guarneri Family", "1626-1744", "Cremona, Italy"], ["Rugeri Family", "1628-1719", "Cremona, Italy"], ["Carlo Bergonzi", "1683-1747", "Cremona, Italy"], ["Jacob Stainer", "1617-1683", "Austria"]]}
{"Top Violinist": [["Name", "Period", "Nationality"], ["Antonio Vivaldi", "1678-1741", "Italian"], ["Niccolo Paganini", "1782-1840", "Italian"], ["Pablo de Sarasate", "1852-1904", "Spainish"], ["Eugene Ysaye", "1858-1931", "Belgian"], ["Fritz Kreisler", "1875-1962", "Astria-American"], ["Jascha Heifetz", "1901-1987", "Russian-American"], ["David Oistrakh", "1908-1974", "Russian"], ["Yehundi Menuhin", "1916-1999", "American"], ["Itzhak Perlman", "1945-", "Israeli-American"], ["Hilary Hahn", "1979-", "American"]]}

Write data

Export an array

Suppose you have the following array:

>>> data = [['G', 'D', 'A', 'E'], ['Thomastik-Infield Domaints', 'Thomastik-Infield Domaints', 'Thomastik-Infield Domaints', 'Pirastro'], ['Silver wound', '', 'Aluminum wound', 'Gold Label Steel']]

And here is the code to save it as an excel file :

>>> p.save_as(array=data, dest_file_name="example.xls")

Let's verify it:

>>> p.get_sheet(file_name="example.xls")
pyexcel_sheet1:
+----------------------------+----------------------------+----------------------------+------------------+
| G                          | D                          | A                          | E                |
+----------------------------+----------------------------+----------------------------+------------------+
| Thomastik-Infield Domaints | Thomastik-Infield Domaints | Thomastik-Infield Domaints | Pirastro         |
+----------------------------+----------------------------+----------------------------+------------------+
| Silver wound               |                            | Aluminum wound             | Gold Label Steel |
+----------------------------+----------------------------+----------------------------+------------------+

And here is the code to save it as a csv file :

>>> p.save_as(array=data,
...           dest_file_name="example.csv",
...           dest_delimiter=':')

Let's verify it:

>>> with open("example.csv") as f:
...     for line in f.readlines():
...         print(line.rstrip())
...
G:D:A:E
Thomastik-Infield Domaints:Thomastik-Infield Domaints:Thomastik-Infield Domaints:Pirastro
Silver wound::Aluminum wound:Gold Label Steel

Export a list of dictionaries

>>> records = [
...     {"year": 1903, "country": "Germany", "speed": "206.7km/h"},
...     {"year": 1964, "country": "Japan", "speed": "210km/h"},
...     {"year": 2008, "country": "China", "speed": "350km/h"}
... ]
>>> p.save_as(records=records, dest_file_name='high_speed_rail.xls')

Export a dictionary of single key value pair

>>> henley_on_thames_facts = {
...     "area": "5.58 square meters",
...     "population": "11,619",
...     "civial parish": "Henley-on-Thames",
...     "latitude": "51.536",
...     "longitude": "-0.898"
... }
>>> p.save_as(adict=henley_on_thames_facts, dest_file_name='henley.xlsx')

Export a dictionary of single dimensonal array

>>> ccs_insights = {
...     "year": ["2017", "2018", "2019", "2020", "2021"],
...     "smart phones": [1.53, 1.64, 1.74, 1.82, 1.90],
...     "feature phones": [0.46, 0.38, 0.30, 0.23, 0.17]
... }
>>> p.save_as(adict=ccs_insights, dest_file_name='ccs.csv')

Export a dictionary of two dimensional array as a book

Suppose you want to save the below dictionary to an excel file :

>>> a_dictionary_of_two_dimensional_arrays = {
...      'Sheet 1':
...          [
...              [1.0, 2.0, 3.0],
...              [4.0, 5.0, 6.0],
...              [7.0, 8.0, 9.0]
...          ],
...      'Sheet 2':
...          [
...              ['X', 'Y', 'Z'],
...              [1.0, 2.0, 3.0],
...              [4.0, 5.0, 6.0]
...          ],
...      'Sheet 3':
...          [
...              ['O', 'P', 'Q'],
...              [3.0, 2.0, 1.0],
...              [4.0, 3.0, 2.0]
...          ]
...  }

Here is the code:

>>> p.save_book_as(
...    bookdict=a_dictionary_of_two_dimensional_arrays,
...    dest_file_name="book.xls"
... )

If you want to preserve the order of sheets in your dictionary, you have to pass on an ordered dictionary to the function itself. For example:

>>> data = OrderedDict()
>>> data.update({"Sheet 2": a_dictionary_of_two_dimensional_arrays['Sheet 2']})
>>> data.update({"Sheet 1": a_dictionary_of_two_dimensional_arrays['Sheet 1']})
>>> data.update({"Sheet 3": a_dictionary_of_two_dimensional_arrays['Sheet 3']})
>>> p.save_book_as(bookdict=data, dest_file_name="book.xls")

Let's verify its order:

>>> book_dict = p.get_book_dict(file_name="book.xls")
>>> for key, item in book_dict.items():
...     print(json.dumps({key: item}))
{"Sheet 2": [["X", "Y", "Z"], [1, 2, 3], [4, 5, 6]]}
{"Sheet 1": [[1, 2, 3], [4, 5, 6], [7, 8, 9]]}
{"Sheet 3": [["O", "P", "Q"], [3, 2, 1], [4, 3, 2]]}

Please notice that "Sheet 2" is the first item in the book_dict, meaning the order of sheets are preserved.

Transcoding

Note

Please note that pyexcel-cli can perform file transcoding at command line. No need to open your editor, save the problem, then python run.

The following code does a simple file format transcoding from xls to csv:

>>> p.save_as(file_name="birth.xls", dest_file_name="birth.csv")

Again it is really simple. Let's verify what we have gotten:

>>> sheet = p.get_sheet(file_name="birth.csv")
>>> sheet
birth.csv:
+-------+--------+----------+
| name  | weight | birth    |
+-------+--------+----------+
| Adam  | 3.4    | 03/02/15 |
+-------+--------+----------+
| Smith | 4.2    | 12/11/14 |
+-------+--------+----------+

Note

Please note that csv(comma separate value) file is pure text file. Formula, charts, images and formatting in xls file will disappear no matter which transcoding tool you use. Hence, pyexcel is a quick alternative for this transcoding job.

Let use previous example and save it as xlsx instead

>>> p.save_as(file_name="birth.xls",
...           dest_file_name="birth.xlsx") # change the file extension

Again let's verify what we have gotten:

>>> sheet = p.get_sheet(file_name="birth.xlsx")
>>> sheet
pyexcel_sheet1:
+-------+--------+----------+
| name  | weight | birth    |
+-------+--------+----------+
| Adam  | 3.4    | 03/02/15 |
+-------+--------+----------+
| Smith | 4.2    | 12/11/14 |
+-------+--------+----------+

Excel book merge and split operation in one line

Merge all excel files in directory into a book where each file become a sheet

The following code will merge every excel files into one file, say "output.xls":

from pyexcel.cookbook import merge_all_to_a_book
import glob


merge_all_to_a_book(glob.glob("your_csv_directory\*.csv"), "output.xls")

You can mix and match with other excel formats: xls, xlsm and ods. For example, if you are sure you have only xls, xlsm, xlsx, ods and csv files in your_excel_file_directory, you can do the following:

from pyexcel.cookbook import merge_all_to_a_book
import glob


merge_all_to_a_book(glob.glob("your_excel_file_directory\*.*"), "output.xls")

Split a book into single sheet files

Suppose you have many sheets in a work book and you would like to separate each into a single sheet excel file. You can easily do this:

>>> from pyexcel.cookbook import split_a_book
>>> split_a_book("megabook.xls", "output.xls")
>>> import glob
>>> outputfiles = glob.glob("*_output.xls")
>>> for file in sorted(outputfiles):
...     print(file)
...
Sheet 1_output.xls
Sheet 2_output.xls
Sheet 3_output.xls

for the output file, you can specify any of the supported formats

Extract just one sheet from a book

Suppose you just want to extract one sheet from many sheets that exists in a work book and you would like to separate it into a single sheet excel file. You can easily do this:

>>> from pyexcel.cookbook import extract_a_sheet_from_a_book
>>> extract_a_sheet_from_a_book("megabook.xls", "Sheet 1", "output.xls")
>>> if os.path.exists("Sheet 1_output.xls"):
...     print("Sheet 1_output.xls exists")
...
Sheet 1_output.xls exists

for the output file, you can specify any of the supported formats

Hidden feature: partial read

Most pyexcel users do not know, but other library users were requesting partial read

When you are dealing with huge amount of data, e.g. 64GB, obviously you would not like to fill up your memory with those data. What you may want to do is, record data from Nth line, take M records and stop. And you only want to use your memory for the M records, not for beginning part nor for the tail part.

Hence partial read feature is developed to read partial data into memory for processing.

You can paginate by row, by column and by both, hence you dictate what portion of the data to read back. But remember only row limit features help you save memory. Let's you use this feature to record data from Nth column, take M number of columns and skip the rest. You are not going to reduce your memory footprint.

Why did not I see above benefit?

This feature depends heavily on the implementation details.

pyexcel-xls (xlrd), pyexcel-xlsx (openpyxl), pyexcel-ods (odfpy) and pyexcel-ods3 (pyexcel-ezodf) will read all data into memory. Because xls, xlsx and ods file are effective a zipped folder, all four will unzip the folder and read the content in xml format in full, so as to make sense of all details.

Hence, during the partial data is been returned, the memory consumption won't differ from reading the whole data back. Only after the partial data is returned, the memory comsumption curve shall jump the cliff. So pagination code here only limits the data returned to your program.

With that said, pyexcel-xlsxr, pyexcel-odsr and pyexcel-htmlr DOES read partial data into memory. Those three are implemented in such a way that they consume the xml(html) when needed. When they have read designated portion of the data, they stop, even if they are half way through.

In addition, pyexcel's csv readers can read partial data into memory too.

Let's assume the following file is a huge csv file:

>>> import datetime
>>> import pyexcel as pe
>>> data = [
...     [1, 21, 31],
...     [2, 22, 32],
...     [3, 23, 33],
...     [4, 24, 34],
...     [5, 25, 35],
...     [6, 26, 36]
... ]
>>> pe.save_as(array=data, dest_file_name="your_file.csv")

And let's pretend to read partial data:

>>> pe.get_sheet(file_name="your_file.csv", start_row=2, row_limit=3)
your_file.csv:
+---+----+----+
| 3 | 23 | 33 |
+---+----+----+
| 4 | 24 | 34 |
+---+----+----+
| 5 | 25 | 35 |
+---+----+----+

And you could as well do the same for columns:

>>> pe.get_sheet(file_name="your_file.csv", start_column=1, column_limit=2)
your_file.csv:
+----+----+
| 21 | 31 |
+----+----+
| 22 | 32 |
+----+----+
| 23 | 33 |
+----+----+
| 24 | 34 |
+----+----+
| 25 | 35 |
+----+----+
| 26 | 36 |
+----+----+

Obvious, you could do both at the same time:

>>> pe.get_sheet(file_name="your_file.csv",
...     start_row=2, row_limit=3,
...     start_column=1, column_limit=2)
your_file.csv:
+----+----+
| 23 | 33 |
+----+----+
| 24 | 34 |
+----+----+
| 25 | 35 |
+----+----+

The pagination support is available across all pyexcel plugins.

Note

No column pagination support for query sets as data source.

Formatting while transcoding a big data file

If you are transcoding a big data set, conventional formatting method would not help unless a on-demand free RAM is available. However, there is a way to minimize the memory footprint of pyexcel while the formatting is performed.

Let's continue from previous example. Suppose we want to transcode "your_file.csv" to "your_file.xls" but increase each element by 1.

What we can do is to define a row renderer function as the following:

>>> def increment_by_one(row):
...     for element in row:
...         yield element + 1

Then pass it onto save_as function using row_renderer:

>>> pe.isave_as(file_name="your_file.csv",
...             row_renderer=increment_by_one,
...             dest_file_name="your_file.xlsx")

Note

If the data content is from a generator, isave_as has to be used.

We can verify if it was done correctly:

>>> pe.get_sheet(file_name="your_file.xlsx")
your_file.csv:
+---+----+----+
| 2 | 22 | 32 |
+---+----+----+
| 3 | 23 | 33 |
+---+----+----+
| 4 | 24 | 34 |
+---+----+----+
| 5 | 25 | 35 |
+---+----+----+
| 6 | 26 | 36 |
+---+----+----+
| 7 | 27 | 37 |
+---+----+----+

Stream APIs for big file : A set of two liners

When you are dealing with BIG excel files, you will want pyexcel to use constant memory.

This section shows you how to get data from your BIG excel files and how to export data to excel files in two lines at most, without eating all your computer memory.

Two liners for get data from big excel files

Get a list of dictionaries

Suppose you want to process the following coffee data again:

Top 5 coffeine drinks:

Coffees	Serving Size	Caffeine (mg)
Starbucks Coffee Blonde Roast	venti(20 oz)	475
Dunkin' Donuts Coffee with Turbo Shot	large(20 oz.)	398
Starbucks Coffee Pike Place Roast	grande(16 oz.)	310
Panera Coffee Light Roast	regular(16 oz.)	300

Let's get a list of dictionary out from the xls file:

>>> records = p.iget_records(file_name="your_file.xls")

And let's check what do we have:

>>> for r in records:
...     print(f"{r['Serving Size']} of {r['Coffees']} has {r['Caffeine (mg)']} mg")
venti(20 oz) of Starbucks Coffee Blonde Roast has 475 mg
large(20 oz.) of Dunkin' Donuts Coffee with Turbo Shot has 398 mg
grande(16 oz.) of Starbucks Coffee Pike Place Roast has 310 mg
regular(16 oz.) of Panera Coffee Light Roast has 300 mg

Please do not forgot the second line to close the opened file handle:

>>> p.free_resources()

Get two dimensional array

Instead, what if you have to use pyexcel.get_array to do the same:

>>> for row in p.iget_array(file_name="your_file.xls", start_row=1):
...     print(f"{row[1]} of {row[0]} has {row[2]} mg")
venti(20 oz) of Starbucks Coffee Blonde Roast has 475 mg
large(20 oz.) of Dunkin' Donuts Coffee with Turbo Shot has 398 mg
grande(16 oz.) of Starbucks Coffee Pike Place Roast has 310 mg
regular(16 oz.) of Panera Coffee Light Roast has 300 mg

Again, do not forgot the second line:

>>> p.free_resources()

where start_row skips the header row.

Data export in one liners

Export an array

Suppose you have the following array:

>>> data = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]

And here is the code to save it as an excel file :

>>> p.isave_as(array=data, dest_file_name="example.xls")

But the following line is not required because the data source are not file sources:

>>> # p.free_resources()

Let's verify it:

>>> p.get_sheet(file_name="example.xls")
pyexcel_sheet1:
+---+---+---+
| 1 | 2 | 3 |
+---+---+---+
| 4 | 5 | 6 |
+---+---+---+
| 7 | 8 | 9 |
+---+---+---+

And here is the code to save it as a csv file :

>>> p.isave_as(array=data,
...            dest_file_name="example.csv",
...            dest_delimiter=':')

Let's verify it:

>>> with open("example.csv") as f:
...     for line in f.readlines():
...         print(line.rstrip())
...
1:2:3
4:5:6
7:8:9

Export a list of dictionaries

>>> records = [
...     {"year": 1903, "country": "Germany", "speed": "206.7km/h"},
...     {"year": 1964, "country": "Japan", "speed": "210km/h"},
...     {"year": 2008, "country": "China", "speed": "350km/h"}
... ]
>>> p.isave_as(records=records, dest_file_name='high_speed_rail.xls')

Export a dictionary of single key value pair

>>> henley_on_thames_facts = {
...     "area": "5.58 square meters",
...     "population": "11,619",
...     "civial parish": "Henley-on-Thames",
...     "latitude": "51.536",
...     "longitude": "-0.898"
... }
>>> p.isave_as(adict=henley_on_thames_facts, dest_file_name='henley.xlsx')

Export a dictionary of single dimensonal array

>>> ccs_insights = {
...     "year": ["2017", "2018", "2019", "2020", "2021"],
...     "smart phones": [1.53, 1.64, 1.74, 1.82, 1.90],
...     "feature phones": [0.46, 0.38, 0.30, 0.23, 0.17]
... }
>>> p.isave_as(adict=ccs_insights, dest_file_name='ccs.csv')
>>> p.free_resources()

Export a dictionary of two dimensional array as a book

Suppose you want to save the below dictionary to an excel file :

>>> a_dictionary_of_two_dimensional_arrays = {
...      'Sheet 1':
...          [
...              [1.0, 2.0, 3.0],
...              [4.0, 5.0, 6.0],
...              [7.0, 8.0, 9.0]
...          ],
...      'Sheet 2':
...          [
...              ['X', 'Y', 'Z'],
...              [1.0, 2.0, 3.0],
...              [4.0, 5.0, 6.0]
...          ],
...      'Sheet 3':
...          [
...              ['O', 'P', 'Q'],
...              [3.0, 2.0, 1.0],
...              [4.0, 3.0, 2.0]
...          ]
...  }

Here is the code:

>>> p.isave_book_as(
...    bookdict=a_dictionary_of_two_dimensional_arrays,
...    dest_file_name="book.xls"
... )

If you want to preserve the order of sheets in your dictionary, you have to pass on an ordered dictionary to the function itself. For example:

>>> from pyexcel._compact import OrderedDict
>>> data = OrderedDict()
>>> data.update({"Sheet 2": a_dictionary_of_two_dimensional_arrays['Sheet 2']})
>>> data.update({"Sheet 1": a_dictionary_of_two_dimensional_arrays['Sheet 1']})
>>> data.update({"Sheet 3": a_dictionary_of_two_dimensional_arrays['Sheet 3']})
>>> p.isave_book_as(bookdict=data, dest_file_name="book.xls")
>>> p.free_resources()

Let's verify its order:

>>> import json
>>> book_dict = p.get_book_dict(file_name="book.xls")
>>> for key, item in book_dict.items():
...     print(json.dumps({key: item}))
{"Sheet 2": [["X", "Y", "Z"], [1, 2, 3], [4, 5, 6]]}
{"Sheet 1": [[1, 2, 3], [4, 5, 6], [7, 8, 9]]}
{"Sheet 3": [["O", "P", "Q"], [3, 2, 1], [4, 3, 2]]}

Please notice that "Sheet 2" is the first item in the book_dict, meaning the order of sheets are preserved.

File format transcoding on one line

Note

Please note that the following file transcoding could be with zero line. Please install pyexcel-cli and you will do the transcode in one command. No need to open your editor, save the problem, then python run.

The following code does a simple file format transcoding from xls to csv:

>>> import pyexcel
>>> p.save_as(file_name="birth.xls", dest_file_name="birth.csv")

Again it is really simple. Let's verify what we have gotten:

>>> sheet = p.get_sheet(file_name="birth.csv")
>>> sheet
birth.csv:
+-------+--------+----------+
| name  | weight | birth    |
+-------+--------+----------+
| Adam  | 3.4    | 03/02/15 |
+-------+--------+----------+
| Smith | 4.2    | 12/11/14 |
+-------+--------+----------+

Note

Please note that csv(comma separate value) file is pure text file. Formula, charts, images and formatting in xls file will disappear no matter which transcoding tool you use. Hence, pyexcel is a quick alternative for this transcoding job.

Let use previous example and save it as xlsx instead

>>> import pyexcel
>>> p.isave_as(file_name="birth.xls",
...            dest_file_name="birth.xlsx") # change the file extension

Again let's verify what we have gotten:

>>> sheet = p.get_sheet(file_name="birth.xlsx")
>>> sheet
pyexcel_sheet1:
+-------+--------+----------+
| name  | weight | birth    |
+-------+--------+----------+
| Adam  | 3.4    | 03/02/15 |
+-------+--------+----------+
| Smith | 4.2    | 12/11/14 |
+-------+--------+----------+

Available Plugins

A list of file formats supported by external plugins

Package name	Supported file formats	Dependencies
pyexcel-io	csv, csvz1, tsv, tsvz2
pyexcel-xls	xls, xlsx(read only), xlsm(read only)	xlrd, xlwt
pyexcel-xlsx	xlsx	openpyxl
pyexcel-ods3	ods	pyexcel-ezodf, lxml
pyexcel-ods	ods	odfpy

Dedicated file reader and writers

Package name	Supported file formats	Dependencies
pyexcel-xlsxw	xlsx(write only)	XlsxWriter
pyexcel-libxlsxw	xlsx(write only)	libxlsxwriter
pyexcel-xlsxr	xlsx(read only)	lxml
pyexcel-xlsbr	xlsb(read only)	pyxlsb
pyexcel-odsr	read only for ods, fods	lxml
pyexcel-odsw	write only for ods	loxun
pyexcel-htmlr	html(read only)	lxml,html5lib
pyexcel-pdfr	pdf(read only)	camelot

Plugin shopping guide

Since 2020, all pyexcel-io plugins have dropped the support for python versions which are lower than 3.6. If you want to use any of those Python versions, please use pyexcel-io and its plugins versions that are lower than 0.6.0.

Except csv files, xls, xlsx and ods files are a zip of a folder containing a lot of xml files

The dedicated readers for excel files can stream read

In order to manage the list of plugins installed, you need to use pip to add or remove a plugin. When you use virtualenv, you can have different plugins per virtual environment. In the situation where you have multiple plugins that does the same thing in your environment, you need to tell pyexcel which plugin to use per function call. For example, pyexcel-ods and pyexcel-odsr, and you want to get_array to use pyexcel-odsr. You need to append get_array(..., library='pyexcel-odsr').

Other data renderers

Package name	Supported file formats	Dependencies	Python versions
pyexcel-text	write only:rst, mediawiki, html, latex, grid, pipe, orgtbl, plain simple read only: ndjson r/w: json	tabulate	2.6, 2.7, 3.3, 3.4 3.5, 3.6, pypy
pyexcel-handsontable	handsontable in html	handsontable	same as above
pyexcel-pygal	svg chart	pygal	2.7, 3.3, 3.4, 3.5 3.6, pypy
pyexcel-sortable	sortable table in html	csvtotable	same as above
pyexcel-gantt	gantt chart in html	frappe-gantt	except pypy, same as above

Footnotes

Acknowledgement

All great work have been done by odf, ezodf, xlrd, xlwt, tabulate and other individual developers. This library unites only the data access code.

License

New BSD License

---

1. zipped csv file↩

2. zipped tsv file↩

pyexcel-ods's Issues

Support modifying cells

The docs imply that this package can only read an ODS file into a Python dictionary or write a Python dictionary to an ODS file. This destroys all data in the ODS file that doesn't have a Python equivalent, such as formatting and formulas. It would be convenient to be able to modify an ODS file in place, such as modifying only certain cells in the file.

Cell Comments are treated like normal text content

When reading a file which contains cells with comments, these comments are added to the text of the cell.

<table:table-cell table:style-name="ce227" office:value-type="string" calcext:value-type="string">
<office:annotation draw:style-name="gr2" draw:text-style-name="P2" svg:width="82.18pt" svg:height="26.62pt" svg:x="94.25pt" svg:y="0pt" draw:caption-point-x="-17.29pt" draw:caption-point-y="14.68pt"><dc:date>2017-08-25T00:00:00</dc:date>
<text:p text:style-name="P1">comment</text:p>
</office:annotation>
<text:p>key</text:p>
</table:table-cell>

The content of this cell after reading with pyexcel-ods will be comment\nkey

I think this is due to the way it is handled in def __read_text_cell(self, cell): link

It simply reads all the elements by type P, and also gets the comment-paragraph from the <office:annotation> element. Afterwards all paragraphs are joined together.

Would it be possible to look for P elements only as direct children of the table-cell?

use "<text:p>" instead of "<text:line-break/>" for newlines (line breaks) in cells

Hi, I wrote a script odsfiles2col.py to combine a bunch of text files into an ods file of two columns, with file names as the key column and file contents as the value column. For example odsfiles2col.py pineapple.txt raspberry.txt would create this file: out.ods . You can see that the line breaks in the text files disappear. So I found this thread and did the following:

mkdir tmp
cd tmp
unzip ../out.ods
perl -i -pe 's#<text:line-break/>#</text:p><text:p>#g' content.xml
zip -r ../fixed.ods .

and the newlines are now correctly shown in the newly created fixed.ods file. Should we do that in the first place when writing out the ods file?

isstream(instance) returns False if StringIO is from cStringIO package

I am using pyexcel_ods.get_data with Flask to parse ODS files uploaded by users as below:

from cStringIO import StringIO
...
io = StringIO()
file_storage = request.files['filename.ods'] 
file_storage.save(io)
data = get_data(io)

(I am not sure this is the correct way to do it. If someone knows a better way, please point me out.)
That will rise:

  File "/env/lib/python2.7/site-packages/pyexcel_ods/__init__.py", line 344, in get_data
    return read_data(afile, file_type=file_type, **keywords)
  File "/env/lib/python2.7/site-packages/pyexcel_io/__init__.py", line 311, in get_data
    data = load_data(afile, file_type=file_type, **keywords)
  File "/env/lib/python2.7/site-packages/pyexcel_io/__init__.py", line 137, in load_data
    raise IOError(MESSAGE_ERROR_03)
IOError: cannot handle unknown content

If I change the first line to:
from StringIO import StringIO
It will work. But from the doc, cStringIO.StringIO is faster than StringIO.StringIO.

"There is no item named 'mimetype' in the archive"

raceback (most recent call last):
  File "parse_outcome.py", line 3, in <module>
    sheet = pe.get_book(file_name="***.ods")
  File "/Library/Python/2.7/site-packages/pyexcel/sources/__init__.py", line 235, in get_book
    book_stream = _get_book(**keywords)
  File "/Library/Python/2.7/site-packages/pyexcel/sources/__init__.py", line 199, in _get_book
    sheets, filename, path = source.get_data()
  File "/Library/Python/2.7/site-packages/pyexcel/sources/file.py", line 87, in get_data
    sheets = load_data(self.file_name, **self.keywords)
  File "/Library/Python/2.7/site-packages/pyexcel_io/__init__.py", line 166, in load_data
    **keywords)
  File "/Library/Python/2.7/site-packages/pyexcel_io/base.py", line 140, in __init__
    self.native_book = self.load_from_file(filename, **keywords)
  File "/Library/Python/2.7/site-packages/pyexcel_ods/__init__.py", line 234, in load_from_file
    return load(filename)
  File "/Library/Python/2.7/site-packages/odf/opendocument.py", line 610, in load
    mimetype = z.read('mimetype')
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 935, in read
    return self.open(name, "r", pwd).read()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 961, in open
    zinfo = self.getinfo(name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 909, in getinfo
    'There is no item named %r in the archive' % name)
KeyError: "There is no item named 'mimetype' in the archive"

Exception shown in simple sheet - currency issue?

The attached causes pyexcel_ods/odsr.py 0.5.2 to throw a TypeError on a simple read:

data = get_data("test.ods")

test.ods.gz

IndexError: list index out of range for file with combined cells

reference: pyexcel/pyexcel#61

Move from pyexcel issue 61 here.

Cannot read from url: AttributeError: addinfourl instance has no attribute 'seek'

import pyexcel as pe

pe.get_book(url="https://github.com/pyexcel/pyexcel-ods/raw/master/tests/fixtures/white_space.ods");

  File "/usr/lib64/python2.7/zipfile.py", line 770, in __init__
    self._RealGetContents()
  File "/usr/lib64/python2.7/zipfile.py", line 809, in _RealGetContents
    endrec = _EndRecData(fp)
  File "/usr/lib64/python2.7/zipfile.py", line 208, in _EndRecData
    fpin.seek(0, 2)
AttributeError: addinfourl instance has no attribute 'seek'

pyexcel/pyexcel-xls#20

the text in last node is read out if a paragraph constains a list of child nodes

It is reproduced.

Reference: pyexcel/pyexcel#81

Empty newlines missing from output

Say I have a spreadsheet with only 1 cell with this data:

Line 1

Line 2

There's a space in the middle, but this happens

>>> from pyexcel_ods import get_data
>>> print(get_data('example.ods')['Sheet1'])
[['Line 1\nLine2']]

I think there should be '\n\n' in there, not '\n'. Thanks

cell with =24/2 formatted as hour results in "ValueError: invalid literal for int() with base 10: 'H0'"

I am trying example from readme and on

data = get_data("test.ods")

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pyexcel_ods/__init__.py", line 344, in get_data
    return read_data(afile, file_type=file_type, **keywords)
  File "/usr/local/lib/python2.7/dist-packages/pyexcel_io/__init__.py", line 314, in get_data
    data[key] = list(data[key])
  File "/usr/local/lib/python2.7/dist-packages/pyexcel_ods/__init__.py", line 184, in to_array
    cell_value = self._read_cell(cell)
  File "/usr/local/lib/python2.7/dist-packages/pyexcel_ods/__init__.py", line 217, in _read_cell
    n_value = VALUE_CONVERTERS[cell_type](value)
  File "/usr/local/lib/python2.7/dist-packages/pyexcel_ods/__init__.py", line 78, in time_value
    minute = int(value[5:7])
ValueError: invalid literal for int() with base 10: 'H0'

To reproduce:

• create a new file
• put =24/2 into A1 cell
• put 23:00 into A2 cell
• copy format from A2 cell to A1 cell
• save as test.ods
• attempt to load with

from pyexcel_ods import get_data
data = get_data("your_file.ods")

Ods file was created with LibreOffice Calc - Version: 5.0.4.2, Build ID: 1:5.0.4rc2-0ubuntu1trusty1, Locale: pl-PL (en_US.UTF-8). Running on Ubuntu 14.04.

Would it be useful to upload this .ods file somewhere or is it reproducible also in other environments?

Does it support Python 3.7

If it does, please update the README.md

odfpy pin problematic

2b33918 added == pin, which causes problems when patch releases occur.

Can't open any ods spreadsheet created in LibreOffice 5.2

Trying to open a ods spreedsheet created in LibreOffice 5.2, even an empty one (see attachment)
empty.zip

variablesSheet = pe.get_book(file_name=self.variablesEntry.get())

I get this error

  File "C:\Users\gibo\AppData\Local\Programs\Python\Python35-32\lib\tkinter\__init__.py", line 3118, in insert
    self.tk.call((self._w, 'insert', index, chars) + args)
  File "C:\Users\gibo\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pyexcel\book.py", line 354, in __str__
    return self.texttable
  File "C:\Users\gibo\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pyexcel\book.py", line 363, in custom_presenter
    self.save_to(memory_source)
  File "C:\Users\gibo\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pyexcel\book.py", line 263, in save_to
    source.write_data(self)
  File "C:\Users\gibo\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pyexcel\sources\file_source_output.py", line 97, in write_data
    **self.keywords)
  File "C:\Users\gibo\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pyexcel\sources\rendererfactory.py", line 53, in render_book_to_stream
    self.render_book(book)
  File "C:\Users\gibo\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pyexcel\sources\rendererfactory.py", line 67, in render_book
    self.render_sheet(sheet)
  File "C:\Users\gibo\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pyexcel\sources\renderer\_texttable.py", line 14, in render_sheet
    self.write_title))
  File "C:\Users\gibo\AppData\Local\Programs\Python\Python35-32\lib\site-packages\pyexcel\sources\renderer\_texttable.py", line 25, in render_text_table
    table.set_cols_dtype(['t'] * len(data[0]))
IndexError: list index out of range

Same code works as expected with existing ods documents created with previous versions of OpenOffice.

pip show pyexcel-ods

---
Metadata-Version: 1.1
Name: pyexcel-ods
Version: 0.2.2

Try to get integer type from a float field

Explore this statement in the reference document:

..The value is stored without a decimal point if the value is an integer;..

at this line: https://github.com/pyexcel/pyexcel-ods/blob/master/pyexcel_ods/__init__.py#L44

reference: pyexcel/pyexcel#32

Formulas are not being evaluated

If you use save_data to write a cell starting with "=", it renders the raw text of the formula and does not evaluate the formula's logic.

>>> from pyexcel_ods import save_data
>>> from collections import OrderedDict
>>> data = OrderedDict()
>>> data.update({"Sheet 1": [
...     ['A', 'B', 'C']] + \
...     [[1, 2, '=A2+B2']]
... })
>>> 
>>> save_data('test.ods', data)

Support for "Open Document (Flat XML) .fodt"

I am using .fodt for version control, pyexcel-ods should be able to open .fodt files (it jut needs to skip the zip/unzip).

IntegerAccuracyLossError on i586

======================================================================
[   68s] ERROR: test_bug_fixes.test_issue_30
[   68s] ----------------------------------------------------------------------
[   68s] Traceback (most recent call last):
[   68s]   File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
[   68s]     self.test(*self.arg)
[   68s]   File "/home/abuild/rpmbuild/BUILD/pyexcel-ods-0.5.4/tests/test_bug_fixes.py", line 149, in test_issue_30
[   68s]     sheet.save_as(test_file)
[   68s]   File "/usr/lib/python2.7/site-packages/pyexcel/internal/meta.py", line 266, in save_as
[   68s]     return save_sheet(self, file_name=filename, **keywords)
[   68s]   File "/usr/lib/python2.7/site-packages/pyexcel/internal/core.py", line 43, in save_sheet
[   68s]     return _save_any(a_source, sheet)
[   68s]   File "/usr/lib/python2.7/site-packages/pyexcel/internal/core.py", line 55, in _save_any
[   68s]     a_source.write_data(instance)
[   68s]   File "/usr/lib/python2.7/site-packages/pyexcel/plugins/sources/file_output.py", line 31, in write_data
[   68s]     self._file_name, sheet, **self._keywords
[   68s]   File "/usr/lib/python2.7/site-packages/pyexcel/plugins/renderers/excel.py", line 27, in render_sheet_to_file
[   68s]     save_data(file_name, data, **keywords)
[   68s]   File "/usr/lib/python2.7/site-packages/pyexcel_io/io.py", line 127, in save_data
[   68s]     **keywords
[   68s]   File "/usr/lib/python2.7/site-packages/pyexcel_io/io.py", line 144, in store_data
[   68s]     writer.write(data)
[   68s]   File "/usr/lib/python2.7/site-packages/pyexcel_io/book.py", line 220, in write
[   68s]     sheet_writer.write_array(incoming_dict[sheet_name])
[   68s]   File "/usr/lib/python2.7/site-packages/pyexcel_io/sheet.py", line 174, in write_array
[   68s]     self.write_row(row)
[   68s]   File "/home/abuild/rpmbuild/BUILD/pyexcel-ods-0.5.4/pyexcel_ods/odsw.py", line 70, in write_row
[   68s]     self.write_cell(row, cell)
[   68s]   File "/home/abuild/rpmbuild/BUILD/pyexcel-ods-0.5.4/pyexcel_ods/odsw.py", line 53, in write_cell
[   68s]     cell = converter_func(cell)
[   68s]   File "/usr/lib/python2.7/site-packages/pyexcel_io/service.py", line 197, in throw_exception
[   68s]     raise exceptions.IntegerAccuracyLossError("%s is too big" % value)
[   68s] IntegerAccuracyLossError: 999999999999999 is too big
[   68s] In order to keep its accuracy, please save as string. Then convert to int, long or float after the value will be read back

https://build.opensuse.org/package/live_build_log/home:jayvdb:pyexcel/python-pyexcel-ods/openSUSE_Tumbleweed/i586

Consecutive blank rows are ignored..

reference: pyexcel/pyexcel-io#8

performance issue with v0.2.1 and v0.2.2

It take too long to read 384kb file. v0.2.0 did not have such a problem. the problem is because it iterate the sheet twice and had to remember repeated cells. it tried to implement random access to any cell but now it turns out that it has a performance penalty.

However, pyexcel-io v.0.3.0 has a new implementation for to_array. so reverting back to v0.2.0 code but using pyexcel-io v0.3.0's new design is the solution.

pyexcel/pyexcel#65

Symption: Problem opening ODS file - Actual problem: excessive trailing columns with empty values

Seems like there may be ODS files that pyexcel-ods struggles with?

eg UKgov housing data (via here).

I can see elements of it in pandas, eg using:

import pandas as pd
pd.DataFrame.from_dict(data,orient='index')

or

pd.DataFrame.from_dict(data)

so it's getting something out...

But if I try to inspect the parsed object or cast it to JSON, it just hangs my python environment (so a deep level of nesting going on in there?).

Also, the data has datetimes in it which causes causes the example json.loads() viewer to fail. I found a simple fix on StackOverflow.

ValueError: invalid literal for int() ...

Howdy.
I have an ODS file in which there is a column with a custom date/time format (yyyy-mm-dd hh:mm). The file was generated by libreoffice.
The xml for a cell in that column is
<table:table-cell table:style-name="ce4" office:value-type="date" office:date-value="2015-08-17T19:20:00" calcext:value-type="date"><text:p>2015-08-17 19:20</text:p></table:table-cell>

So, when loading that file, I get the error
File "/usr/local/lib/python2.7/dist-packages/pyexcel_ods/__init__.py", line 56, in date_value day = int(tokens[2]) ValueError: invalid literal for int() with base 10: '17T19:10:00'
because the code is expecting the data to be in the format yyyy-mm-dd.

The safest bet would be to always use dateutil.parser.parse(value) which handles the iso format.

Repeated cells in ODS file are blank

When reading an ODS file and a cell has repeated values, pyexcel-ods will give the repeated cells an empty value.
The problem is in ODSSheet.to_array:

repeat = cell.getAttribute("numbercolumnsrepeated")
if(not repeat):
  has_value = True
  ret = self._read_cell(cell)
  arrCells.append(ret)
else:
  r = int(repeat)
  for i in range(0, r):
    arrCells.append("")

Is should be:

repeat = cell.getAttribute("numbercolumnsrepeated")
if(not repeat):
  has_value = True
  ret = self._read_cell(cell)
  arrCells.append(ret)
else:
  r = int(repeat)
  ret = self._read_cell(cell) # Fixed
  for i in range(0, r):
    arrCells.append(ret) # Fixed

Error saving datetime

Hi,
I have ordered dict like
OrderedDict([(u'Foglio1', [[u'COGNOME', u'NOME', u'C.F.', u'****',.....],
[u'', u'', u'********', 18, 164848, datetime.datetime(2018, 7, 11, 0, 0),....]])

When I save, boot with sheet.save_to_memory("ods", io) and save_data(io, data) result exception
'datetime.datetime' object has no attribute 'split'

Variable | Value
cell | datetime.datetime(2018, 7, 11, 0, 0)
cell_odf_type | 'string'
cell_odf_value_token | 'value'
cell_to_be_written | <odf.element.Element instance at 0x7f3ce43a15f0>
cell_type | <type 'datetime.datetime'>
converter_func | None
row | <odf.element.Element instance at 0x7f3ce439d7e8>
self | <pyexcel_ods.odsw.ODSSheetWriter object at 0x7f3ce4959d50>

If I save with sheet.save_to_memory("xls", io) it works .

Am I wrong or it is a bug ?
How can I use datetime ?

Issue attempting to save single-sheet file

Just installed pyexcel-ods 0.1.1 via pip (which auto-installed the dependancies pyexcel-io 0.1.0 and odfpy 0.9.6)

Attempted to run the sample code from the site - specifically I attempted to use both the 'read from a file' and 'write to a file' code in one script, altering the data in between. My script was as follows:

    from pyexcel_ods import save_data, get_data

    gnsFile = "C:\\users\\main1\\documents\\gns 16-05-16.ods"

    data = get_data(gnsFile)
    data[0].append('Test')
    save_data(gnsFile, data)

However that resulted in the following error:

Traceback (most recent call last):
  File "GNSDataRetriever.py", line 11, in <module>
    save_data(gnsFile, data)
  File "C:\Python27\lib\site-packages\pyexcel_ods\__init__.py", line 335, in sav
e_data
    write_data(afile, data, file_type=file_type, **keywords)
  File "C:\Python27\lib\site-packages\pyexcel_io\__init__.py", line 260, in stor
e_data
    writer.write(data)
  File "C:\Python27\lib\site-packages\pyexcel_io\base.py", line 286, in write
    keys = sheet_dicts.keys()
AttributeError: 'list' object has no attribute 'keys'

I was able to work around it by changing the body of the write method in pyexcel_io/base.py to the following:

        try:
            keys = sheet_dicts.keys()
            for name in keys:
                sheet = self.create_sheet(name)
                if sheet is not None:
                    sheet.write_array(sheet_dicts[name])
                    sheet.close()
        except AttributeError:
            sheet = self.create_sheet('Sheet1')
            if sheet is not None:
                sheet.write_array(sheet_dicts)
                sheet.close()

(Of course it would be more correct to try to actually find out the actual sheet name and write it out as that rather than just Sheet1 - but for my use case it doesn't matter).

I haven't done any further testing (and am not likely to have to time to anytime soon, sadly), but it looks like the data object passed ends up being only a dict if there are multiple sheets, with each value in the dict being a list, but if there is only one sheet the list is all that gets passed, rather than a dict with one key and value. If I do get the time I'll test out that theory with a simple multi-sheet file.

I can provide a copy of the specific file that I noticed this issue with, if required - it doesn't contain anything particularly sensitive.

The version of libreoffice used to originally create the file was:

Version: 5.0.4.2
Build ID: 2b9802c1994aa0b7dc6079e128979269cf95bc78
Locale: en-AU (en_AU)

on windows.

Python version is:

Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:32:19) [MSC v.1500 32 bit (Intel)] on win32

Write Image

Is possible to write image on cells?
Thank you

Faulty unicode conversion when saving ods in environments with ASCII as default encoding

Hi,

I ran into the beloved

UnicodeEncodeError: 'ascii' codec can't encode character u'\xd6' in position 1: ordinal not in range(128)

when saving data containing non ASCII characters to ods.

Python 2.6.6 on CentOS 6, kernel 2.6.32-504.16.2.el6.x86_64
pyexcel version: 0.1.7

A specific test code that triggers the problem is the following:

#!/usr/bin/python
# -*- encoding: utf-8 -*-
import pyexcel as pe
from pyexcel.ext import ods

data = [[1,2,3],['Joe', unicode('DÖE', 'utf-8', 'strict'), 'Name']]
my_sheet = pe.Sheet(data)
my_sheet.save_as('test.ods')

A quick analysis showed that the type is recognized correctly, but somehow the conversion gets called twice when setting the attributes, once for the value-type, which specifies the type correctly as string, and then again for the value, which uses cnv_double, which breaks because of the non ASCII characters. I stopped debugging from there, as I would need a more thorough understanding of the employed conversion mechanisms.

Here is the full traceback of the error:

Traceback (most recent call last):
File "odstest.py", line 8, in
my_sheet.save_as('test.ods')
File "/usr/lib/python2.6/site-packages/pyexcel/sheets/sheet.py", line 39, in save_as
return self.save_to(source)
File "/usr/lib/python2.6/site-packages/pyexcel/sheets/sheet.py", line 33, in save_to
source.write_data(self)
File "/usr/lib/python2.6/site-packages/pyexcel/sources/file.py", line 35, in write_data
w.write_reader(sheet)
File "/usr/lib/python2.6/site-packages/pyexcel/writers.py", line 75, in write_reader
self.write_rows(to_array(reader))
File "/usr/lib/python2.6/site-packages/pyexcel/writers.py", line 45, in write_rows
self.writer.write_array(table)
File "/usr/lib/python2.6/site-packages/pyexcel_io/base.py", line 222, in write_array
self.write_row(r)
File "/usr/lib/python2.6/site-packages/pyexcel_ods/init.py", line 257, in write_row
self.write_cell(tr, x)
File "/usr/lib/python2.6/site-packages/pyexcel_ods/init.py", line 246, in write_cell
tc.setAttrNS(OFFICENS, x_odf_value_token, x)
File "/usr/lib/python2.6/site-packages/odf/element.py", line 440, in setAttrNS
self.attributes[(namespace, localpart)] = c.convert((namespace, localpart), value, self)
File "/usr/lib/python2.6/site-packages/odf/attrconverters.py", line 1489, in convert
return conversion(attribute, value, element)
File "/usr/lib/python2.6/site-packages/odf/attrconverters.py", line 77, in cnv_double
return str(arg)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xd6' in position 1: ordinal not in range(128)

and here is the output of an added print statement in attrconverters.py:

print attribute, value, element

(u'urn:oasis:names:tc:opendocument:xmlns:office:1.0', u'mimetype') application/vnd.oasis.opendocument.spreadsheet
(u'urn:oasis:names:tc:opendocument:xmlns:office:1.0', u'version') 1.1
(u'urn:oasis:names:tc:opendocument:xmlns:table:1.0', u'name') pyexcel
(u'urn:oasis:names:tc:opendocument:xmlns:office:1.0', 'value-type') float
(u'urn:oasis:names:tc:opendocument:xmlns:office:1.0', 'value') 1
(u'urn:oasis:names:tc:opendocument:xmlns:office:1.0', 'value-type') float
(u'urn:oasis:names:tc:opendocument:xmlns:office:1.0', 'value') 2
(u'urn:oasis:names:tc:opendocument:xmlns:office:1.0', 'value-type') float
(u'urn:oasis:names:tc:opendocument:xmlns:office:1.0', 'value') 3
(u'urn:oasis:names:tc:opendocument:xmlns:office:1.0', 'value-type') string
(u'urn:oasis:names:tc:opendocument:xmlns:office:1.0', 'value') Joe
(u'urn:oasis:names:tc:opendocument:xmlns:office:1.0', 'value-type') string
(u'urn:oasis:names:tc:opendocument:xmlns:office:1.0', 'value') DÖE

I would greatly appreciate your help on solving this issue.
Thanks for the awesome package anyways.

Best regards,
Felix

Sheet.save_as ("output.ods") doesn't support long() values

Got exception : AttributeError: 'long' object has no attribute 'split'

When running the code :

import pyexcel
sheet = pyexcel.Sheet()
sheet [0,0] = 123456678912345678901234567
sheet.save_as ("output.ods")

Seems to be due to pyexcel_io.service.ODS_WRITE_FORMAT_CONVERSION dictionnary,
that is not aware of long() datatype

File : pyexecl_ods/odsw.py

cell_odf_type = converter.ODS_WRITE_FORMAT_CONVERSION.get(cell_type, "string")
...
if cell_odf_type != 'string':
...
else:
lines = cell.split('\n')

Whereas long() values doesn't support split() -> exception

I would suggest to add long in ODS_WRITE_FORMAT_CONVERSION
Or cell.spit("\n") in a try...catch block

Python 3.4 causes travis fails

Thanks for your project and thanks to support .ods
It seems there is a single point which put your branch red

Travis answer is:

This lxml version requires Python 2.7, 3.5 or later.

May be you can consider to remove support of python 3.4

You're project could be more visible with a green branch

Thanks

Don't work save-data import

Hi!
I just updated to the versions:

pyexcel-ods 0.5.6
puexcel-io 0.5.20

And now when importing "store_sata" I get this error:

from pyexcel_ods import save_data, get_data
File "/usr/lib/python3.7/site-packages/pyexcel_ods/init.py", line 12, in
from pyexcel_io.io import store_data as write_data
ImportError: cannot import name 'store_data' from 'pyexcel_io.io' (/usr/lib/python3.7/site-packages/pyexcel_io/io.py)

Open ODS from page request, exception Unicode

Resolved with ods3

date presentation error

pyexcel/pyexcel#164

this library failed to write date type so that they get displayed in libre-office properly

Incremental writing of rows is very slow

I'm using the following software:

$ sudo -H pip3 list | egrep 'odfpy|ods'
odfpy                         1.4.1               
pyexcel-ods                   0.6.0               

and the following test program:

#!/bin/env python3
from datetime import datetime

from pyexcel_io import writer

FILENAME = 'tmp.ods'
LIBRARIES = ['pyexcel-ods', 'pyexcel-ods3', 'pyexcel-odsw']
LIBRARY = LIBRARIES[0]
ROW = ['c' + str(c) for c in range(153)]


with writer.Writer('ods', library=LIBRARY) as file_stream:
    file_stream.open(FILENAME)
    page_stream = file_stream.writer.create_sheet('mysheet')
    #
    # Write 10, 100, 1000, ... rows.
    #
    row_count = 10
    for multiplier in range(4):
        before = datetime.utcnow()
        for row in range(row_count):
            ROW[0] = 'r' + str(row)
            page_stream.write_row(ROW)
        after = datetime.utcnow()
        print('row_count', row_count, 'seconds', (after - before).total_seconds())
        before = after
        row_count *= 10
    page_stream.close()
    after = datetime.utcnow()
    print('page_stream close', 'seconds', (after - before).total_seconds())
    before = after
after = datetime.utcnow()
print('file_stream close', 'seconds', (after - before).total_seconds())
before = after

and see the following output:

$ ./ods.py 
row_count 10 seconds 0.019138
row_count 100 seconds 0.191849
row_count 1000 seconds 2.262572
row_count 10000 seconds 23.984489
page_stream close seconds 5.154405
file_stream close seconds 27.650097

As you can see, it takes about 26s to write_row() 11110 times, and then about 33s to close the page_stream and file_stream. Is this as expected? Is there anything I can do to speed it up? I actually need to get to a row_count somewhere circa 200k (and possibly more), and that is clearly going to take quite a while with these numbers.

Support merged cells

Let's say I have a .ods file with merged cells like this:

+----------------------+------------+----------+----------+
| Header row, column 1 | Header 2   | Header 3 | Header 4 |
+======================+============+==========+==========+
| body row 1, column 1 | column 2   | column 3 | column 4 |
+----------------------+------------+----------+----------+
| body row 2           | Cells may span columns.          |
+----------------------+----------------------------------+
| body row 3           | Cells may  | - Table cells       |
+----------------------+ span rows. | - contain           |
| body row 4           |            | - body elements.    |
+----------------------+------------+---------------------+

Then if I do:

import pyexcel_ods

data = pyexcel_ods.get_data('example.ods')
pyexcel_ods.save_data('example.ods', data)

The resulting table loses it's merged cells and looks like this:

+----------------------+-------------------------+------------------+----------+
| Header row, column 1 | Header 2                | Header 3         | Header 4 |
+======================+=========================+==================+==========+
| body row 1, column 1 | column 2                | column 3         | column 4 |
+----------------------+-------------------------+------------------+----------+
| body row 2           | Cells may span columns. |                  |          |
+----------------------+-------------------------+------------------+----------+
| body row 3           | Cells may               | - Table cells    |          |
|                      | span rows.              | - contain        |          |
|                      |                         | - body elements. |          |
+----------------------+-------------------------+------------------+----------+
| body row 4           |                         |                  |          |
+----------------------+-------------------------+------------------+----------+

Is there any way to maintain merged cells? Thanks.

data type presentation concerning ods/openoffice calc files

Basically the same issue as mentioned here:
#31

just that I work with openoffice cal, therefor ods files. (and not libreoffice as in the case mentioned there)
and any Datetime.date object saved to the ods file
is saved as a number, not as a proper date.

for example, if I have 25.03.2021 as a datetime.date object in the code and I try to save it as a cell content in the ods file,
it will be saved as 45133 in the openoffice calc file.

for information, I use windows 10.
I have already tried updating via pip install --upgrade pyexcel-ods3 but there seems to be no newer version availlable.

Lost dependence

In version 0.5.0, the "pyexcel" dependency is required to work correctly, but pip does not install it.

I have needed to install pyexcel to write ods files.

Multi line cell value is flattened

Hi,

I'm trying to read a multi line cell value, but I'm getting a single line string.
What I'm getting:

single line	multi line
correct	thisshouldbemultiline

What should be:

single line	multi line
correct	this should be multi line

Saving in xls format and using pyexcel-xls works, but I need to support ods files too.

Thanks!