Giter Site home page Giter Site logo

pyhdb's Introduction

SAP HANA Database Client for Python

Important Notice

image

This public repository is read-only and no longer maintained.

The active maintained alternative is SAP HANA Python Client: https://pypi.org/project/hdbcli/

A pure Python client for the SAP HANA Database based on the SAP HANA Database SQL Command Network Protocol.

pyhdb supports Python 2.7, 3.3, 3.4, 3.5, 3.6 and also PyPy on Linux, OSX and Windows. It implements a large part of the DBAPI Specification v2.0 (PEP 249).

Table of contents

Install

Install from Python Package Index:

$ pip install pyhdb

Install from GitHub via pip:

$ pip install git+https://github.com/SAP/pyhdb.git

You can also install the latest version direct from a cloned git repository.

$ git clone https://github.com/SAP/pyhdb.git
$ cd pyhdb
$ python setup.py install

Getting started

If you do not have access to a SAP HANA server, go to the SAP HANA Developer Center and choose one of the options to get your own trial SAP HANA Server.

For using PyHDB with hanatrial instance, follow this guide.

The basic pyhdb usage is common to database adapters implementing the DBAPI 2.0 interface (PEP 249). The following example shows how easy it's to use the pyhdb module.

>>> import pyhdb
>>> connection = pyhdb.connect(
    host="example.com",
    port=30015,
    user="user",
    password="secret"
)

>>> cursor = connection.cursor()
>>> cursor.execute("SELECT 'Hello Python World' FROM DUMMY")
>>> cursor.fetchone()
(u"Hello Python World",)

>>> connection.close()

Establish a database connection

The function pyhdb.connect creates a new database session and returns a new Connection instance. Please note that port isn't the instance number of you SAP HANA database. The SQL port of your SAP HANA is made up of 3<instance-number>15 for example the port of the default instance number 00 is 30015.

Currently pyhdb only supports the user and password authentication method. If you need another authentication method like SAML or Kerberos than please open a GitHub issue. Also there is currently no support of encrypted network communication between client and database.

Cursor object

With the method cursor of your Connection object you create a new Cursor object. This object is able to execute SQL statements and fetch one or multiple rows of the resultset from the database.

Example select

>>> cursor = connection.cursor()
>>> cursor.execute("SELECT SCHEMA_NAME, TABLE_NAME FROM TABLES")

After you executed a statement you can fetch one or multiple rows from the resultset.

>>> cursor.fetchone()
(u'SYS', u'DUMMY')

>>> cursor.fetchmany(3)
[(u'SYS', u'DUMMY'), (u'SYS', u'PROCEDURE_DATAFLOWS'), (u'SYS', u'PROCEDURE_MAPPING')]

You can also fetch all rows from your resultset.

>>> cursor.fetchall()
[(u'SYS', u'DUMMY'), (u'SYS', u'PROCEDURE_DATAFLOWS'), (u'SYS', u'PROCEDURE_MAPPING'), ...]

Example Create table

With the execute method you can also execute DDL statements like CREATE TABLE.

>>> cursor.execute('CREATE TABLE PYHDB_TEST("NAMES" VARCHAR (255) null)')

Example insert

You can also execute DML Statements with the execute method like INSERT or DELETE. The Cursor attribute rowcount contains the number of affected rows by the last statement.

>>> cursor.execute("INSERT INTO PYHDB_TEST VALUES('Hello Python World')")
>>> cursor.rowcount
1

LOBs

Three different types of LOBs are supported and corresponding LOB classes have been implemented: * Blob - binary LOB data * Clob - string LOB data containing only ascii characters * NClob - string (unicode for Python 2.x) LOB data containing any valid unicode character

LOB instance provide a file-like interface (similar to StringIO instances) for accessing LOB data. For HANA LOBs lazy loading of the actual data is implemented behind the scenes. An initial select statement for a LOB only loads the first 1024 bytes on the client:

>>> mylob = cursor.execute('select myclob from mytable where id=:1', [some_id]).fetchone()[0]
>>> mylob
<Clob length: 2000 (currently loaded from hana: 1024)>

By calling the read(<num-chars>)-method more data will be loaded from the database once <num-chars> exceeds the number of currently loaded data:

>>> myload.read(1500)   # -> returns the first 1500 chars, by loading extra 476 chars from the db
>>> mylob
<Clob length: 2000 (currently loaded from hana: 1500)>
>>> myload.read()   # -> returns the last 500 chars by loading them from the db
>>> mylob
<Clob length: 2000 (currently loaded from hana: 2000)>

Using the seek() methods it is possible to point the file pointer position within the LOB to arbitrary positions. tell() will return the current position.

LOBs can be written to the database via insert or update-statemens with LOB data provided either as strings or LOB instances:

>>> from pyhdb import NClob
>>> nclob_data = u'朱の子ましける日におえつかうまつ'
>>> nclob = NClob(nclob_data)
>>> cursor.execute('update mynclob set nclob_1=:1, nclob_2=:2 where id=:3', [nclob, nclob_data, myid])

Note

Currently LOBs can only be written in the database for sizes up to 128k (entire amount of data provided in one update or insert statement). This constraint will be removed in one of the next releases of PyHDB. This limitation does however not apply when reading LOB data from the database.

Stored Procedures

Rudimentary support for Stored Procedures call, scalar parameters only:

The script shall call the stored procedure PROC_ADD2 (source below):

>>> sql_to_prepare = 'call PROC_ADD2 (?, ?, ?, ?)'
>>> params = {'A':2, 'B':5, 'C':None, 'D': None}
>>> psid = cursor.prepare(sql_to_prepare)
>>> ps = cursor.get_prepared_statement(psid)
>>> cursor.execute_prepared(ps, [params])
>>> result = cursor.fetchall()
>>> for l in result:
>>>     print l

from the stored procedure:

create procedure PROC_ADD2 (in a int, in b int, out c int, out d char)
language sqlscript
reads sql data as
begin
    c := :a + :b;
    d := 'A';
end

Transaction handling

Please note that all cursors created from the same connection are not isolated. Any change done by one cursor is immediately visible to all other cursors from same connection. Cursors created from different connections are isolated as the connection based on the normal transaction handling.

The connection objects provides to method commit which commit any pending transaction of the connection. The method rollback undo all changes since the last commit.

Contribute

If you found bugs or have other issues than you are welcome to create a GitHub Issue. If you have questions about usage or something similar please create a Stack Overflow Question with tag pyhdb.

Run tests

pyhdb provides a test suite which covers the most use-cases and protocol parts. To run the test suite you need the pytest and mock package. Afterwards just run py.test inside of the root directory of the repository.

$ pip install pytest mock
$ py.test

You can also test different python version with tox.

$ pip install tox
$ tox

Tracing

For debugging purposes it is sometimes useful to get detailed tracing information about packages sent to hana and those received from the database. There are two ways to turn on the print out of tracing information:

  1. Set the environment variable HDB_TRACING=1 before starting Python, e.g. (bash-syntax!):
$ HDB_TRACE=1 python
  1. Import the pyhdb module and set pyhdb.tracing = True

Then perform some statements on the database and enjoy the output.

To get tracing information when running pytest provide the -s option:

$ HDB_TRACE=1 py.test -s

ToDos

  • Allow execution of stored database procedure
  • Support of SELECT FOR UPDATE
  • Authentication methods
    • SAML
    • Kerberos

pyhdb's People

Contributors

bsrdjan avatar firnkes avatar gbandet avatar jarus avatar mathebox avatar ralhei avatar spartan057 avatar timogasda avatar vovasty avatar xchrdw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pyhdb's Issues

Python and HCP Trial connectivity issue.

I am trying to establish a connection with my HCP trail account from python but facing the below issue:

capture

The code snippet i used to establish this connection is :

import dbapi

conn = dbapi.connect( 'hanatrial.ondemand.com' , 30015, 'p1942054738', 'xxxxxx')

print ( conn.isconnected())

Please note :

  • i have placed this code under python folder of hdbclient.

  • I have copied these files into the Lib folder of python

          * __init__.py, dbapi.py, resultrow.py
    
          * pyhdbcli.pdb, pyhdbcli.pyd
    

Please let me know what am i missing to establish a connection between HCP and python.

PyHDB doesn't work with Jython 2.7.0

Here is exception, which I get

>>> connection = pyhdb.connect(host="hana", port=30215, user="user", password="password")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/fedor/jenv/Lib/site-packages/pyhdb/__init__.py", line 30, in connect
    conn.connect()
  File "/Users/fedor/jenv/Lib/site-packages/pyhdb/connection.py", line 141, in connect
    agreed_auth_part = self._auth_manager.perform_handshake()
  File "/Users/fedor/jenv/Lib/site-packages/pyhdb/auth.py", line 50, in perform_handshake
    response = self.connection.send_request(request)
  File "/Users/fedor/jenv/Lib/site-packages/pyhdb/connection.py", line 84, in send_request
    return self.__send_message_recv_reply(payload.getvalue())
  File "/Users/fedor/jenv/Lib/site-packages/pyhdb/connection.py", line 98, in _Connection__send_message_recv_reply
    header = ReplyMessage.header_from_raw_header_data(raw_header)
  File "/Users/fedor/jenv/Lib/site-packages/pyhdb/connection.py", line 98, in _Connection__send_message_recv_reply
    header = ReplyMessage.header_from_raw_header_data(raw_header)
  File "/Users/fedor/jenv/Lib/site-packages/pyhdb/protocol/message.py", line 107, in header_from_raw_header_data
    raise Exception("Invalid message header received")
Exception: Invalid message header received

Same call works with normal Python

close connection problem

this is python 3.5.1
work with following sample
import pyhdb
connection02 = pyhdb.connect(
host="ykfl00620460a.amer.global.corp.sap",
port=30215,
user="i826608",
password="Initial1"
)
curs02 = connection02.cursor()
curs02.execute("SELECT 'Hello Python World' FROM DUMMY")
str1 = curs02.fetchone()[0]
curs02.close()
connection02.close()

it would fail with error -
Error
Traceback (most recent call last):
File "C:\Apps\Python35\lib\site-packages\pyhdb\connection.py", line 107, in __send_message_recv_reply
_payload = self._socket.recv(header.payload_length - payload.tell())
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Test\Hana\hrtt\test0.py", line 39, in test_hana
hana_test()
File "C:\Test\Hana\hrtt\shared\fun_hrtt.py", line 606, in hana_test
connection02.close()
File "C:\Apps\Python35\lib\site-packages\pyhdb\connection.py", line 168, in close
reply = self.send_request(request)
File "C:\Apps\Python35\lib\site-packages\pyhdb\connection.py", line 84, in send_request
return self.__send_message_recv_reply(payload.getvalue())
File "C:\Apps\Python35\lib\site-packages\pyhdb\connection.py", line 121, in __send_message_recv_reply
raise OperationalError("Lost connection to HANA server (%r)" % error)
pyhdb.exceptions.OperationalError: Lost connection to HANA server (ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

but the following case would work fine:
import pyhdb
connection02 = pyhdb.connect(
host="ykfl00620460a.amer.global.corp.sap",
port=30215,
user="i826608",
password="Initial1"
)
curs02 = connection02.cursor()
curs02.execute("SELECT 'Hello Python World' FROM DUMMY")
curs02.fetchone()
curs02.close()
connection02.close()

  • I did not read data with:
    str1 = curs02.fetchone()

How to fetch data from HANA in Python in table format

Hi

I have installed the Pyhdb and everything correctly. However the problem I see is that everytime i need to fetch the data it comes as a tuple. I want to fetch the data as a pandas dataframe. Like in Postgressql if I connect it with Python and use an adapter called pysopg2 on the connection object like below:

`Creating database connection string
In [9]:
USER_CRED_FILE = os.path.join(os.path.expanduser('~'), '.dslab_user.cred')
def fetchDBCredentials(dbcred_file=USER_CRED_FILE):
"""
Read database access credentials from the file in $HOME/.ipynb_dslab.cred
"""
#Read database credentials from user supplied file
conf = ConfigParser.ConfigParser()
conf.read(dbcred_file)
#host, port, user, database, password
host = conf.get('database_creds','host')
port = conf.get('database_creds','port')
user = conf.get('database_creds','user')
database = conf.get('database_creds','database')
password = conf.get('database_creds','password')

#Initialize connection string
conn_str =  """dbname='{database}' user='{user}' host='{host}' port='{port}' password='{password}'""".format(                       
                database=database,
                host=host,
                port=port,
                user=user,
                password=password
        )
return conn_str

Create a connection object to the database
In [10]:
conn = psycopg2.connect(fetchDBCredentials())

`

Then I can simply run

conn.commit

or I can do

psql.read_sql(sql,conn)

and it would get the result in table format.

Please see the below two links to get a sense what I am asking.

https://github.com/vatsan/gp_jupyter_notebook_templates/blob/master/notebooks/00_database_connectivity_setup.ipynb

https://github.com/vatsan/gp_jupyter_notebook_templates

How do I get that in HANA. Something similar to Pyscopg2 I guess?

Insert Execution into a series table doesnot work

Hi all,

If you create a table like this:
Create column table Series ("READING_TYPE_ID" NVARCHAR(10) NOT NULL ,"TIMESTAMP" LONGDATE CS_LONGDATE NOT NULL ,"VALUE" DOUBLE CS_DOUBLE) SERIES (SERIES KEY("READING_TYPE_ID") PERIOD FOR SERIES ("TIMESTAMP") EQUIDISTANT INCREMENT BY INTERVAL .000001 SECOND MISSING ELEMENTS ALLOWED)

And then do insert like this:
insertStmnt = 'insert into ' + table + ' values ' + '( ' + row["READING_TYPE_ID"] + " , '" + row["TIMESTAMP"] + "' , " + row["VALUE"] + ' )'
cursor.execute(insertStmnt)

The statement gets executed with 0 exit code but the value is not inserted. With jaydebeapi and jpype it works though. Fix this.

Connection error

Receiving a interesting error when trying to do a insert into SPS12 from Python 3.5:
Traceback (most recent call last):
File "C:\Users\sap\Anaconda3\lib\site-packages\pyhdb-0.3.1-py3.5.egg\pyhdb\connection.py", line 108, in __send_message_recv_reply
_payload = self._socket.recv(header.payload_length - payload.tell())
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "Coon_test.py", line 21, in
connection.close()
File "C:\Users\sap\Anaconda3\lib\site-packages\pyhdb-0.3.1-py3.5.egg\pyhdb\connection.py", line 168, in close
reply = self.send_request(request)
File "C:\Users\sap\Anaconda3\lib\site-packages\pyhdb-0.3.1-py3.5.egg\pyhdb\connection.py", line 84, in send_request
return self.__send_message_recv_reply(payload.getvalue())
File "C:\Users\sap\Anaconda3\lib\site-packages\pyhdb-0.3.1-py3.5.egg\pyhdb\connection.py", line 121, in __send_message_recv_reply
raise OperationalError("Lost connection to HANA server (%r)" % error)
pyhdb.exceptions.OperationalError: Lost connection to HANA server (ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

Any thoughts?

ConfigParser.NoSectionError: No Section: 'formatters'

I installed pyhdb via cloning the git and executing the setup.py.
While importing pyhdb the following error appears:

>>> import pyhdb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pyhdb-0.2.1-py2.7.egg/pyhdb/__init__.py", line 20, in <module>
    logging.config.fileConfig(os.path.join(this_dir, 'logging.conf'))
  File "/usr/lib/python2.7/logging/config.py", line 77, in fileConfig
    formatters = _create_formatters(cp)
  File "/usr/lib/python2.7/logging/config.py", line 113, in _create_formatters
    flist = cp.get("formatters", "keys")
  File "/usr/lib/python2.7/ConfigParser.py", line 607, in get
    raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'formatters''

When starting the python console in the pyhdb directory and importing pyhdb there, there is no problem.

print is used in pyhdb/protocol/parts.py

print is used in pyhdb/protocol/parts.py so every execution with parameters yield an output

pyhdb/protocol/parts.py
337:        # print repr(payload)
349:        # print 'realobreply called with args', args
372:        # print 'realobreply unpack data called with args', len(lobdata), is_data_included, is_last_data
541:            # print 'row', num_rows, humanhexlify(payload.read())[:500]
555:        print 'writing lob buffers', row_lobs
563:            print 'Writing lob data %d bytes of %d' % (max_data_to_write, lob_buffer.encoded_lob_size)
567:                print 'adding lob_buffer to unwritte_lobs list'

Support of binary locator (type 31)

I want to perform multiple inserts with a prepared statement. The table definition contains a column of ST_POINT. On execution I get the following error:

InterfaceError: Prepared statement parameter datatype not supported: 31

It there any documentation available that would help me to add this feature? Or do you plan to implement it in the near future?

PyHDB does not install for Python 3

Hi,
The command : $ pip install pyhdb installs it only for Python 2.x

On my Raspberry Pi I have both Python 2 and 3.
The example code works on Python 2 but does not work in Python 3. It says : "No module named pyhdb"

How can we easily install it for Python 3 ?

'create local temporary column table #parameter' statement issue

i am try to create a temporary table in hana db using cursor.execute() statement using the following sql statement " create local temporary column table #parameter".
the cursor.fetchall() result is " Require execute() first"

and when i tried to access the temporary table ,its showing an error
INFO: invalid table name: Could not find table/view #PARAMETERS in schema SYSTEM: line 1 col 36 (at pos 35)

but when i tried to execute the statement via hana studio console,the temporary table is getting created.

two clarification

  1. why i am getting "require execute() first" exception
  2. is it possible to create local temporary column table via pyhdb.since i assume that temporary table is available with latest hana version.

Could you please help me out here

Request: Support `qmark` and and `named` as paramstyles

Currently, only format is supported as a paramstyle. Unfortunately, this is some kind of the opposite from the hdbcli python library from the HANA-Client, which supports qmark and named as parameter styles. This makes switching unhandy. A possible abstraction from the underlying connector would have to handle this.

I would like to have more paramstyles supported, here.

connect to tenant database fails

while try to connect to a tenant db in hana like:
connection = pyhdb.connect(
host="example.com",
port=30015,
user="user",
password="secret",
database='db1'
)

it fails with error as -
TypeError: connect() got an unexpected keyword argument 'database'

note that replacing "database" with "dbname" would fail too, any tips?

Support for Timestamp, Time and Date

When trying to insert a datetime value with the following statement

params = [0, datetime.datetime.now()]
cursor.executemany('INSERT INTO TEST_DATETIME VALUES (?, ?)', params)
connection.commit()

this error is thrown

Traceback (most recent call last):
  File "datetime-test.py", line 32, in <module>
    cursor.execute('INSERT INTO TEST_DATETIME VALUES (?, ?)', params)
  File "/Users/max/virtualenvs/hana_load/lib/python2.7/site-packages/pyhdb-0.2.3-py2.7.egg/pyhdb/cursor.py", line 254, in execute
    self.executemany(statement, parameters=[parameters])
  File "/Users/max/virtualenvs/hana_load/lib/python2.7/site-packages/pyhdb-0.2.3-py2.7.egg/pyhdb/cursor.py", line 278, in executemany
    self.execute_prepared(prepared_statement, parameters)
  File "/Users/max/virtualenvs/hana_load/lib/python2.7/site-packages/pyhdb-0.2.3-py2.7.egg/pyhdb/cursor.py", line 185, in execute_prepared
    response = self.connection.send_request(request)
  File "/Users/max/virtualenvs/hana_load/lib/python2.7/site-packages/pyhdb-0.2.3-py2.7.egg/pyhdb/connection.py", line 83, in send_request
    payload = message.pack()  # obtain BytesIO instance
  File "/Users/max/virtualenvs/hana_load/lib/python2.7/site-packages/pyhdb-0.2.3-py2.7.egg/pyhdb/protocol/message.py", line 54, in pack
    self.build_payload(payload)
  File "/Users/max/virtualenvs/hana_load/lib/python2.7/site-packages/pyhdb-0.2.3-py2.7.egg/pyhdb/protocol/message.py", line 44, in build_payload
    segment.pack(payload, commit=self.autocommit)
  File "/Users/max/virtualenvs/hana_load/lib/python2.7/site-packages/pyhdb-0.2.3-py2.7.egg/pyhdb/protocol/segments.py", line 109, in pack
    self.build_payload(payload)
  File "/Users/max/virtualenvs/hana_load/lib/python2.7/site-packages/pyhdb-0.2.3-py2.7.egg/pyhdb/protocol/segments.py", line 95, in build_payload
    part_payload = part.pack(remaining_size)
  File "/Users/max/virtualenvs/hana_load/lib/python2.7/site-packages/pyhdb-0.2.3-py2.7.egg/pyhdb/protocol/parts.py", line 100, in pack
    arguments_count, payload = self.pack_data()
  File "/Users/max/virtualenvs/hana_load/lib/python2.7/site-packages/pyhdb-0.2.3-py2.7.egg/pyhdb/protocol/parts.py", line 408, in pack_data
    pfield = _DataType.prepare(value)
AttributeError: type object 'Timestamp' has no attribute 'prepare'

The class method prepare is missing for the type object Timestamp to convert the value into binary format. The same applies for the type object Time.
The type object Date implements the prepare method. But the value is not properly stored in the database.


Create statement for the table TEST_DATETIME:

CREATE TABLE TEST_DATETIME (ID INTEGER, dt TIMESTAMP);

'Invalid column name' error when using max() in query

When using the max() function in a query, I receive the following error:

>>> cursor.execute('SELECT packages.id AS packages_id, packages.name AS packages_name,
packages.version AS packages_version, packages.enabled AS packages_enabled,
max(packages.version) AS max_1 FROM packages')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/python3.5/site-packages/pyhdb/cursor.py", line 258, in execute
    self._execute_direct(statement)
  File "/lib/python3.5/site-packages/pyhdb/cursor.py", line 217, in _execute_direct
    reply = self.connection.send_request(request)
  File "/lib/python3.5/site-packages/pyhdb/connection.py", line 84, in send_request
    return self.__send_message_recv_reply(payload.getvalue())
  File "/lib/python3.5/site-packages/pyhdb/connection.py", line 124, in __send_message_recv_reply
    return ReplyMessage.unpack_reply(header, payload)
  File "/lib/python3.5/site-packages/pyhdb/protocol/message.py", line 92, in unpack_reply
    segments=tuple(ReplySegment.unpack_from(payload, expected_segments=header.num_segments)),
  File "/lib/python3.5/site-packages/pyhdb/protocol/segments.py", line 152, in unpack_from
    raise error.parts[0].errors[0]
pyhdb.exceptions.DatabaseError: invalid column name: The column 'PACKAGES.ID' is invalid in
the select list because the GROUP BY clause or an aggregation function does not contain it:
line 1 col 8 (at pos 7)

The same query works correctly without the function, as does using it without selecting any other columns. Am I doing something wrong, or is this a bug?

Create stored procedure PROC_ADD2 in unit test

@bsrdjan
Can you create the above mentioned stored procedure when running 'test_call_stored.py' (as a fixture)?
Otherwise this unit tests will fail for users running tests on HCP or their own Hana instance as this stored procedure does of course not exist in those DBs.
Thanks!

stored procedures: table as output

What should work

It is possible to have tables as an output argument for stored procedures:

create procedure PROC_DUMMY (in a int, in b int, out c int, out d DUMMY, out e TABLES)
  language sqlscript
  reads sql data as
  begin
    c := :a + :b;
    d = select * from DUMMY;
    e = select * from TABLES;
  end

Why it fails

Currently calling such a stored procedure fails, since the number of arguments the database expects is 2 but pyhdb expects 4:

# Cursor.next
if len(parameters) != len(self._params_metadata):
    raise ProgrammingError("Prepared statement parameters expected %d supplied %d." %
                           (len(self._params_metadata), len(parameters)))

Removing this check unveils a second issue. It is possible for one cursor to access several result sets. The above example provides three table like results. c is send via OutputParameters and made available through fetchmany by setting self._buffer to these values. The other two (d and e) are proper result sets. Though e never gets fetched since pyhdb assumes that every cursor only operates on one result set. Falsely it believes that having finished d there is no work left to do.

#Cursor.fetchmany
     request = RequestMessage.new(
            self.connection,
            RequestSegment(
                message_types.FETCHNEXT,
                (ResultSetId(self._resultset_id), FetchSize(size - cnt))
            )
        )

How to fix

There are two issues to tackle. The easy one is to fix the expectation that the len of expected parameters by the database is the same as the provided one.

Now to the hard one:

Using cursor.execute_prepared is bad. Python's dbapi proposes the use of .callproc instead. The problem is that the return value of a normal SQL query is a single table, whilst stored procedures can provide zero to many tables. Instead callproc could return two values, a collection of the scalar values and a collection of tables. Both can be empty.

values, tables = cursor.callproc('PROC_DUMMY')

Further support for multiple result sets per cursor has to be implemented.

Inform: @mathebox

pyhdb DEBUG output when running tests with nosetests

Hi all,

I am currently using pyhdb in an application that is also tested with nose. Unfortunately, I get a lot of debug output from pyhdb, making my own output harder to read, especially if many tests use a database connection.

Is there any way to disable pyhdb's debug messages?

Thanks,

Christian

Cannot connect on Windows 64bit

When trying to connect on Windows 64bit with either 32bit or 64bit Python 2.7, the client just blocks indefinitely because it is stuck on a recv call. It is waiting on the server's answer on the AUTHENTICATE message.

This is due to the fact that you are using this struct for the Message header:

struct = struct.Struct('liIIhb9B')

The first parameter is the session ID, which is supposed to be I8 (8 byte integer). But the l you are using here only produces a 4 byte integer on 64bit Windows. If you change this to q it works just fine on Windows, and I have tested this on Linux as well.

batch insert problem

this is python 3.5.1, when try to insert batch data like:
curs01.execute("""select ID,EN,FR,DE,JA,ZH,KO,ZH_HK,ZH_TW,CJK,SPAIN,POLISH,PT,ITALY,
NORWAY,SWEFIN,CP874,CP932,CP936_GB2312_2K,CP950,CP1250,CP1251,
CP1252,CP1253,CP1254,CP1256,CP1255,CP1257,CP1258
from SAMP_LANG_KEYWORD order by ID """)
keywords01 = curs01.fetchall()
for key01 in keywords01:
cmdstr01 = """ INSERT into 24F3AA18E176403CBF3A6111C2B4DF45.SAMP_LANG_KEYWORD
(ID1,EN,FR,DE,JA,ZH,KO,ZH_HK,ZH_TW,CJK,SPAIN,POLISH,PT,ITALY,
NORWAY,SWEFIN,CP874,CP932,CP936,CP950,CP1250,CP1251,
CP1252,CP1253,CP1254,CP1256,CP1255,CP1257,CP1258) VALUES (
:1,:2,:3,:4,:5,:6,:7,:8,:9,:10,
:11,:12,:13,:14,:15,:16,:17,:18,:19,:20,
:21,:22,:23,:24,:25,:26,:27,:28,:29
) """
curs02.execute(cmdstr01,[key01[0],key01[1],key01[2],key01[3],key01[4],key01[5],key01[6],key01[7],key01[8],key01[9],
key01[10],key01[11],key01[12],key01[13],key01[14],key01[15],key01[16],key01[17],key01[18],key01[19],
key01[20],key01[21],key01[22],key01[23],key01[24],key01[25],key01[26],key01[27],key01[28]])

after running above statement, there is no data in the table, I have to add one more line:
curs02.execute('commit')

with above "commit", I could see data, but at the same time, it shows error message like:
Error
Traceback (most recent call last):
File "C:\Test\Hana\hrtt\test0.py", line 39, in test_hana
hana_test(curs1)
File "C:\Test\Hana\hrtt\shared\fun_hrtt.py", line 579, in hana_test
curs02.execute('commit')
File "C:\Apps\Python35\lib\site-packages\pyhdb\cursor.py", line 258, in execute
self._execute_direct(statement)
File "C:\Apps\Python35\lib\site-packages\pyhdb\cursor.py", line 231, in _execute_direct
raise InterfaceError("Invalid or unsupported function code received: %d" % function_code)
pyhdb.exceptions.InterfaceError: Invalid or unsupported function code received: 11

not so sure, what is wrong here

EXECUTION_COUNT in M_SQL_PLAN_CACHE is not updated

Hi PyHDB folks,

I tried to know the execution time of my query.
I read M_SQL_PLAN_CACHE, because it contains TOTAL_EXECUTION_TIME and EXECUTION_COUNT.

First time I tried below code.

STAT = """SELECT TOTAL_EXECUTION_TIME, EXECUTION_COUNT
FROM M_SQL_PLAN_CACHE
WHERE TO_VARCHAR(STATEMENT_STRING)=:1
AND IS_VALID = 'TRUE'"""

q = 'SELECT * FROM EC_TEST_TABLE'

con = pyhdb.connect(host=HOST, password=PASSWORD, user=USER, port=PORT)
cur = con.cursor()

def fetch(q):
    cur.execute(q, [])
    cur.fetchall()
    cur.execute(STAT, [q])
    return cur.fetchall()

A fetch function get execution count and execution time from M_SQL_PLAN_CACHE after executing query and fetching all data.
I thought it would work and it worked if EC_TEST_TABLE table does not have lob field.
But if it has a lob field(either NCLob, CLob or Blob) and the table is not empty, the execution count and time does not increases.

In the second try, I closed cursor before read M_SQL_PLAN_CACHE

def cursor_close_and_fetch(q):
    global cur
    cur.execute(q)
    cur.fetchall()
    cur.close()
    cur = con.cursor()
    cur.execute(STAT, [q])
    return cur.fetchall()

But it does not worked neither.

Finally, I found a workaround, execution time is updated when the connection is closed. But I think closing cursor should be enough to update execution time.

def connection_close_and_fetch(q):
    global cur, con
    cur.execute(q)
    cur.fetchall()
    cur.close()
    con.close()
    con = pyhdb.connect(host=HOST, password=PASSWORD, user=USER, port=PORT)
    cur = con.cursor()
    cur.execute(STAT, [q])
    return cur.fetchall()

UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 5915: character maps to <undefined>

Maybe an issue is not related to PyHDB.

When installing with pip I get the following:


C:\Python34\Scripts\pip run on 06/15/15 15:34:00
Downloading/unpacking pyhdb
Getting page https://pypi.python.org/simple/pyhdb/
URLs to search for versions for pyhdb:

UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 5915: character maps to

CESU8 tests fail on Windows 64bit and OS X Yosemite (narrow Python)

Almost all CESU-8 tests fail for me because of the following error:

ValueError: unichr() arg not in range(0x10000) (narrow Python build)

I think most Python builds are "narrow" build, which means that they do not support 4-byte unicode chracters directly. Instead, they use a surrogate UTF-16 pair (see here and here).

No module named lib.tracing

import pyhdb
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "pyhdb/__init__.py", line 23, in <module>
   from pyhdb.connection import Connection
 File "pyhdb/connection.py", line 22, in <module>
   from pyhdb.auth import AuthManager
 File "pyhdb/auth.py", line 24, in <module>
   from pyhdb.protocol.message import RequestMessage
 File "pyhdb/protocol/message.py", line 21, in <module>
   from pyhdb.lib.tracing import trace
ImportError: No module named lib.tracing

Seems that the lib-package is not included in the repository.

Commit: b50c790

error: [Errno 110] Connection timed out

Hi, i need connect a HCP but, I not can

In [1]: import pyhdb

In [2]: conn = pyhdb.connect(host="hanatrial.ondemand.com", port=30015, user="p1941325981trial",  password="123123")
---------------------------------------------------------------------------
error                                     Traceback (most recent call last)
<ipython-input-2-8dd7eaabb1b9> in <module>()
----> 1 conn = pyhdb.connect(host="hanatrial.ondemand.com", port=30015, user="p1941325981trial",  password="Yw&nDRK4JEs4u")

/usr/local/lib/python2.7/dist-packages/pyhdb-0.1.0-py2.7.egg/pyhdb/__init__.pyc in connect(host, port, user, password, autocommit)
      8 def connect(host, port, user, password, autocommit=False):
      9     connection = Connection(host, port, user, password, autocommit)
---> 10     connection.connect()
     11     return connection

/usr/local/lib/python2.7/dist-packages/pyhdb-0.1.0-py2.7.egg/pyhdb/client.pyc in connect(self)
    110                 return
    111 
--> 112             self._open_socket_and_init_protocoll()
    113 
    114             # Perform the authenication handshake and get the part

/usr/local/lib/python2.7/dist-packages/pyhdb-0.1.0-py2.7.egg/pyhdb/client.pyc in _open_socket_and_init_protocoll(self)
     42     def _open_socket_and_init_protocoll(self):
     43         self._socket = socket.create_connection(
---> 44             (self.host, self.port), self._timeout
     45         )
     46 

/usr/lib/python2.7/socket.pyc in create_connection(address, timeout, source_address)
    569 
    570     if err is not None:
--> 571         raise err
    572     else:
    573         raise error("getaddrinfo returns an empty list")

error: [Errno 110] Connection timed out

In [3]: 

thank for you help!

Some sql 'case' statements fail with numeric parameter substitution

Change 9dc77f6 (using paramstyle 'numeric' over 'format') seems to cause problems with same sql 'case' statements when containing parameter substitution:

query = '''
SELECT DISTINCT group_id,
                max(CASE
                    WHEN m.name = 'Aaron' THEN 4
                    WHEN m.name = 'Ronald' THEN 3
                    WHEN m.name = 'Jack' THEN 2
                    ELSE 0
                END)
FROM sa_example_main m
GROUP BY m.group_id
ORDER BY max(CASE
    WHEN m.name = 'Aaron' THEN 4
    WHEN m.name = 'Ronald' THEN 3
    WHEN m.name = 'Jack' THEN 2
    ELSE :1
END) DESC
'''
params = (0, )
cur.execute(query, params)

results in:

pyhdb.exceptions.DatabaseError: invalid column name: not included in SELECT DISTINCT column list: M.NAME: line 12 col 10 (at pos 317)

If 'paramstyle' equals 'format' (or the :1 in the else statement is replaced with '0') then the command succeeds.

Missing support for TEXT datatype?

When trying to insert data into a TEXT field I'm getting:

InterfaceError: Prepared statement parameter datatype not supported: 32

MVE:

from contextlib import closing
import pyhdb
hanadata = dict(
    user     = '',
    password = '',
    host     = '',
    port     = 30015)

with closing(pyhdb.connect(**hanadata)) as con:
    curs = con.cursor()
    curs.execute('SET SCHEMA MYSCHEMA;')
    curs.execute('CREATE COLUMN TABLE MYTESTTABLE(ID INT PRIMARY KEY, SOMETEXT TEXT);')
    curs.execute('INSERT INTO MYTESTTABLE (ID, SOMETEXT) VALUES (?,?);', [1,'asdf'])
    # won't reach this line
    curs.execute('SELECT * FROM MYTESTTABLE;')
    print(curs.fetchall())

Import Error (ConfigParser) on Python 3.4.3 OS X 10.10.4

I installed pyhdb via pip, got v.0.2.3 and included it as usual.

When including the library the following error comes up:

Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 23 2015, 02:52:03) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyhdb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/pyhdb/pyhdb/__init__.py", line 18, in <module>
    import ConfigParser
ImportError: No module named 'ConfigParser'

Do I have to install dependencies or such? Python finds a "configparser" module (lowercase), but not the ConfigParser.

Thanks in advance!

pyhdb.exceptions.DatabaseError: invalid name of function or procedure: PROC_ADD2

import pyhdb
sql_to_prepare = 'call PROC_ADD2 (?, ?, ?, ?)'
params = {'A':2, 'B':5, 'C':None, 'D': None}
psid = cursor.prepare(sql_to_prepare)
ps = cursor.get_prepared_statement(psid)
cursor.execute_prepared(ps, [params])
result = cursor.fetchall()
for l in result:
print l

me da el siguiente error:
raise error.parts[0].errors[0]
pyhdb.exceptions.DatabaseError: invalid name of function or procedure: PROC_ADD2: line 1 col 6 (at pos 5)

In Hana studio have:
create procedure PROC_ADD2 (in a int, in b int, out c int, out d char)
language sqlscript
begin
c := :a + :b;
d := 'A';
end

Always unpack prepared statement_id

In the SAP HANA studio you see the decimal id of a prepared statement. We should also unpack the statement id to make it easy to use them in the HANA studio for debug purposes.

socket.error: [Errno 61] Connection refused

Hi All,
I tried to connect to my SAP HANA AWS, host is my Public DNS, port 30015, user and password as ones I connect the server from eclipse, but get following error:

Traceback (most recent call last):
  File "<stdin>", line 5, in <module>
  File "/Library/Python/2.7/site-packages/pyhdb/__init__.py", line 30, in connect
    conn.connect()
  File "/Library/Python/2.7/site-packages/pyhdb/connection.py", line 137, in connect
    self._open_socket_and_init_protocoll()
  File "/Library/Python/2.7/site-packages/pyhdb/connection.py", line 66, in _open_socket_and_init_protocoll
    self._socket = socket.create_connection((self.host, self.port), self._timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 575, in create_connection
    raise err
socket.error: [Errno 61] Connection refused

Any help is appreciated!

Thanks!

Warnings are treated as errors

I have a procedure in HANA that, when I run it in HANA Studio runs successfully but returns a warning:

java.sql.SQLWarning: Not recommended feature: DDL statement is used in Dynamic SQL (current dynamic_sql_ddl_error_level = 1)

Unfortunately I do need to use dynamic SQL in that case, so I was planning to ignore the warning.
However, when I try to run the same procedure using PyHDB, my script fails:

_$ python /opt/workflow/scripts/hana/call_procedure_env.py "flatten_sales_territories()" prod
Traceback (most recent call last):
File "/opt/workflow/scripts/hana/call_procedure_env.py", line 26, in
cursor_h.execute(sql)
File "/usr/local/lib/python2.7/dist-packages/pyhdb/cursor.py", line 258, in execute
self._execute_direct(statement)
File "/usr/local/lib/python2.7/dist-packages/pyhdb/cursor.py", line 217, in _execute_direct
reply = self.connection.send_request(request)
File "/usr/local/lib/python2.7/dist-packages/pyhdb/connection.py", line 84, in send_request
return self.__send_message_recv_reply(payload.getvalue())
File "/usr/local/lib/python2.7/dist-packages/pyhdb/connection.py", line 124, in _send_message_recv_reply
return ReplyMessage.unpack_reply(header, payload)
File "/usr/local/lib/python2.7/dist-packages/pyhdb/protocol/message.py", line 92, in unpack_reply
segments=tuple(ReplySegment.unpack_from(payload, expected_segments=header.num_segments)),
File "/usr/local/lib/python2.7/dist-packages/pyhdb/protocol/segments.py", line 150, in unpack_from
raise Exception("Rows affected %s" % (error.parts[0].values,))
Exception: Rows affected (65934,)

Is it possible to allow the library to ignore warnings?

dynamic get table‘s data

for example, I want get the data, but I don't know the tablename. so I need to write the stament like following:
QueryData = 'select * from :1;'
curWrite.execute(QueryColumn, [tableName]).fetchall();
but colsole return "incorrect syntax near ":1""

Pyhdb.connect fails because of Unknown Option Type

Hi all,

It threw the following error while connecting:

Traceback (most recent call last):
File "", line 5, in
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyhdb/init.py", line 10, in connect
connection.connect()
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyhdb/client.py", line 126, in connect
ConnectOptions(DEFAULT_CONNECTION_OPTIONS)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyhdb/protocol/base.py", line 87, in send
return self.connection._send_message(self.pack())
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyhdb/client.py", line 96, in _send_message
return Message.unpack_reply(self, header, payload)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyhdb/protocol/base.py", line 98, in unpack_reply
payload, expected_segments=header[4]
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyhdb/protocol/base.py", line 216, in unpack_from
yield ReplySegment.unpack(segment_header, segment_payload)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyhdb/protocol/base.py", line 232, in unpack
tuple(Part.unpack_from(payload, expected_parts=header[2]))
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyhdb/protocol/base.py", line 307, in unpack_from
part_header[2], part_payload
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyhdb/protocol/parts.py", line 111, in unpack_data
raise Exception("Unknown option type %s" % typ)
Exception: Unknown option type 30

Please fix this.

Support Windows Authentification

Currently a large number of enterprise environments use windows authentification to control access to HANA environments. As such user credentials are not available.

Supporting Windows AD as other drives such as PYODBC etc do would greatly facilitate the usefulness of HANA

Connection issue after upgrade from SPS10 to SPS12

The pyhdb module worked great with Python 3.5.0 (Anaconda distro) on a Windows 2008 server running HANA SPS10. After upgrading to SPS12, we see the following connection error:

OperationalError: Lost connection to HANA server (ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', none, 10054, none)).

We are able to telnet into the server on port 30015 and connect via studio on the same system with no issues. If I put in a 'wrong' username/password I get an authentication error...so the connection is at least making it that far.

Greatly appreciate any assistance from the pyhdb community.

Unhandled error caused NoneType has no encode error

This is caused instering "String" into VARCHAR(1)

  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 874, in commit
    self.transaction.commit()
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 461, in commit
    self._prepare_impl()
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
    self.session.flush()
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2139, in flush
    self._flush(objects)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2259, in _flush
    transaction.rollback(_capture_exception=True)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2223, in _flush
    flush_context.execute()
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
    rec.execute(self)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
    uow
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
    mapper, table, insert)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 835, in _emit_insert_statements
    execute(statement, params)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1396, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/pyhdb/cursor.py", line 260, in execute
    self.executemany(statement, parameters=[parameters])
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/pyhdb/cursor.py", line 284, in executemany
    self.execute_prepared(prepared_statement, parameters)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/pyhdb/cursor.py", line 189, in execute_prepared
    reply = self.connection.send_request(request)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/pyhdb/connection.py", line 83, in send_request
    payload = message.pack()  # obtain BytesIO instance
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/pyhdb/protocol/message.py", line 55, in pack
    self.build_payload(payload)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/pyhdb/protocol/message.py", line 45, in build_payload
    segment.pack(payload, commit=self.autocommit)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/pyhdb/protocol/segments.py", line 94, in pack
    self.build_payload(payload)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/pyhdb/protocol/segments.py", line 80, in build_payload
    part_payload = part.pack(remaining_size)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/pyhdb/protocol/parts.py", line 103, in pack
    arguments_count, payload = self.pack_data(remaining_size - self.header_size)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/pyhdb/protocol/parts.py", line 509, in pack_data
    lob_buffer = LobBuffer(value, _DataType, lob_header_pos)
  File "/Users/michaelkuty/projects/stories/lib/python3.5/site-packages/pyhdb/protocol/parts.py", line 448, in __init__
    enc_data = orig_data.encode()
AttributeError: 'NoneType' object has no attribute 'encode'

proposed solution in #82

Connection Failed unknow option type 30

Connection fails against HANA Rev 8x and 91.
Python 2.6 (r26:66714, May 6 2011, 15:10:21)
[GCC 4.3.4 [gcc-4_3-branch revision 152973]] on linux2

>>> import pyhdb
>>> connection = pyhdb.connect(   host="remote-host...",    port=30415,    user="SYSTEM",    password="....")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib64/python2.6/site-packages/pyhdb-0.1.0-py2.6.egg/pyhdb/__init__.py", line 24, in connect
    connection.connect()
  File "/usr/local/lib64/python2.6/site-packages/pyhdb-0.1.0-py2.6.egg/pyhdb/client.py", line 140, in connect
    ConnectOptions(DEFAULT_CONNECTION_OPTIONS)
  File "/usr/local/lib64/python2.6/site-packages/pyhdb-0.1.0-py2.6.egg/pyhdb/protocol/base.py", line 101, in send
    return self.connection._send_message(self.pack())
  File "/usr/local/lib64/python2.6/site-packages/pyhdb-0.1.0-py2.6.egg/pyhdb/client.py", line 110, in _send_message
    return Message.unpack_reply(self, header, payload)
  File "/usr/local/lib64/python2.6/site-packages/pyhdb-0.1.0-py2.6.egg/pyhdb/protocol/base.py", line 112, in unpack_reply
    payload, expected_segments=header[4]
  File "/usr/local/lib64/python2.6/site-packages/pyhdb-0.1.0-py2.6.egg/pyhdb/protocol/base.py", line 230, in unpack_from
    yield ReplySegment.unpack(segment_header, segment_payload)
  File "/usr/local/lib64/python2.6/site-packages/pyhdb-0.1.0-py2.6.egg/pyhdb/protocol/base.py", line 246, in unpack
    tuple(Part.unpack_from(payload, expected_parts=header[2]))
  File "/usr/local/lib64/python2.6/site-packages/pyhdb-0.1.0-py2.6.egg/pyhdb/protocol/base.py", line 321, in unpack_from
    part_header[2], part_payload
  File "/usr/local/lib64/python2.6/site-packages/pyhdb-0.1.0-py2.6.egg/pyhdb/protocol/parts.py", line 125, in unpack_data
    raise Exception("Unknown option type %s" % typ)
Exception: Unknown option type 30

Rounding error on Timestamp column

I encountered the error "DatabaseError(u'invalid DATE, TIME or TIMESTAMP value: not a valid millisecond: 60000: type_code=16, index=2',)" when running through the Django tutorial https://docs.djangoproject.com/en/1.10/intro/tutorial02/. Specifically, I encountered it when I executed the instruction: "Question.objects.get(pub_date__year=current_year)". The problem is that that instruction generates a query with two parameters to identify the current year: '2017-01-01 00:00:00', and '2017-12-31 23:59:59.999999'. The preparation of the second parameter causes the problem. You don't need Django to reproduce the problem, but I mention it because it shows that the bug, though an edge case, unfortunately shows up in a very well travelled place.

The problem is in the handling of microseconds in pyhdb.protocols.types.Timestamp. The 59 seconds and 999999 microseconds of the second parameter is being rounded up to 60000 in the Timestamp.prepare function. 60000 is, of course, an invalid millisecond value.

I have attached a zip file, sql_parm_test.zip, that containing three files:

  • sql_parm_test.sql: This contains the sql needed to run the test.
  • sql_parm_test.py: This contains the python test needed to reproduce the problem.
  • sql_parm_fix.txt: This is a code snippet that shows the fix that I implemented for my environment.

I am using the following versions of things:

  • HANA, express edition 2.00.000.00.1479874437
  • python 2.7.10
  • pyhdb 0.3.2
  • six 1.10.0
    For the record, I am using the following Django modules:
  • Django 1.10.5
  • django-hana 1.1

sql_parm_test.zip

Regards,

Carl Soane

Keep connection open/Reconnect

Hello,
we're using a couple of long running Python scripts which each have a connection to a HANA instance. We've seen several problems with timeout errors and I wanted to ask if there is a possibility to have something like an "keep connection open" or "automatically reconnect" flag?

Or is it possible to implement that?

MemoryError in connect

I have used PyHDB successfully to connect to a HANA database. I am currently trying to connect to a different database and I get a MemoryError:

>>> import pyhdb
>>> pyhd.connect(host='xxx',port=xxxx,user='xxxx',password='xxxxxx')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/etc/lib64/anaconda/lib/python2.7/site-packages/pyhdb/__init__.py", line 30, in connect
    conn.connect()
  File "/etc/lib64/anaconda/lib/python2.7/site-packages/pyhdb/connection.py", line 141, in connect
    agreed_auth_part = self._auth_manager.perform_handshake()
  File "/etc/lib64/anaconda/lib/python2.7/site-packages/pyhdb/auth.py", line 50, in perform_handshake
    response = self.connection.send_request(request)
  File "/etc/lib64/anaconda/lib/python2.7/site-packages/pyhdb/connection.py", line 84, in send_request
    return self.__send_message_recv_reply(payload.getvalue())
  File "/etc/lib64/anaconda/lib/python2.7/site-packages/pyhdb/connection.py", line 107, in __send_message_recv_reply
    _payload = self._socket.recv(header.payload_length - payload.tell())
MemoryError

Any idea what this failure means?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.