Comments (4)
MAPI 10 support has been removed from the server side so we will never merge this
from pymonetdb.
ah really? why? it was supposed to be faster right (since binary)
from pymonetdb.
It was a good idea but as far as i know the implementation was never completed beyond the proof of concept level. No clients used it, not even mclient/libmapi. The server side implementation copy pasted a lot of existing code and left modified duplicates that were not back-integrated with the original. Over the years the code started to rot and at some point it was decided to remove it. Some of it has been removed, some of it still lingers.
Since then there has been a new development, "COLUMNAR_MODE" that also passes binary data. IIRC it is used in some kind of server-server communication (but not REMOTE TABLE, that is another barrel of fish). I think @aris-koning knows more about it.
We have been talking about adding support for it to client libraries but one of the blockers is that it does not support partial transfers (reply_size) and i don't know if and how it supports the more esoteric data types.
from pymonetdb.
Just to give a bit more background: the implementation of remote tables is based on the remote
MAL module. That module offers a MAL interface to create server-to-server connection, move around binary BAT files and MAL programs between the servers where the latter can be executed as a remote procedures. The original use case for this module is indeed the REMOTE TABLE
feature. However I have done some work on the remote
module, by having it more integrated into the MAPI protocol where it is accessible under the columnar_protocol
flag:
MapiMsg
mapi_set_columnar_protocol(Mapi mid, bool columnar_protocol)
{
if (mid->columnar_protocol == columnar_protocol)
return MOK;
mid->columnar_protocol = columnar_protocol;
if (!mid->connected)
return MOK;
if (columnar_protocol)
return mapi_Xcommand(mid, "columnar_protocol", "1");
else
return mapi_Xcommand(mid, "columnar_protocol", "0");
}
I have used PROTOCOL_COLUMNAR
to implement the first version of remote connection in monetdbe
. I choose to base this on the remote
module as PROTOCOL_10
did not seem to be as complete or used in the core monetdbe code base. While the remote module was already way better tested because of its use in the implementation of REMOTE TABLE
.
But PROTOCOL_COLUMNAR
also requires some scrutiny and probably some improvement:
For once it does serialize vheap
based BAT files like string columns. Also the interaction with the MAPI replay feature needs some attention. And there is probably more stuff if we dig deeper. But I think it first requires a use case before we go deeper there. Maybe better communication performance in pymonetdb would be a nice use case. If so, we might want to change the title of this issue.
from pymonetdb.
Related Issues (20)
- Separate SQL errors from connection errors during query execution HOT 5
- SQLSTATE error code removed from error messages HOT 6
- Fix master branch, travis is failing HOT 1
- Re-enable travis integration HOT 1
- Support day interval type HOT 4
- `cursor.executemany` does not work as expected HOT 2
- Use of BokenPipeError breaks DBAPI compatability in 1.4.0 HOT 2
- Rename pymonetdb (back) to (python-)monetdb HOT 1
- Day interval issue? HOT 1
- Connection object leaks the password HOT 1
- slow data transfer HOT 2
- Github actions broken HOT 3
- Make code style checking stricter (mypy and flake8)
- Value error in dates HOT 1
- Problem with parameters in execute. HOT 5
- Control.status() crashes when connecting without a host name
- Add support for prepared statements HOT 4
- Properly handle multiple result sets HOT 2
- Make Cursor.execute() return None instead of -1 HOT 1
- pymonetdb can not upload to remote MonetDB server from client HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pymonetdb.