davidhamann/python-fmrest

Python wrapper around the FileMaker Data API

python-fmrest's Issues

additional info on data types and their formats when creating records

As I did not find anything in the examples and not even on Claris' website, it took me a while to figure out the correct (?) way to write date, time and timestamps to new records.

So maybe worth an additional (to be created) 'create_data.ipynb' in the examples directory?

Possibly along the lines of:

rec_created = fms.create_record({'mytext': 'Mars',
'mynum': 667,
'mydate': '2/22/2022', # month/day/year
'mytime': '9:15:55 PM', # alternative 24h format
'mystamp': '2/22/2022 9:15:55 PM',
})
print(rec_created)

rec_created2 = fms.create_record( {} ) # empty dict
print(rec_created2)

new_data = {
'mytext': 'Venus',
'mynum': 1234567,
'mydate': '02/22/2022', # month/day/year, with leading zero
'mytime': '21:15:55', # alternative format, no 'AM/PM', see above
'mystamp': '2/22/2022 21:15:55',
}
rec_created3 = fms.create_record(new_data)
print(rec_created3)

does V16 work with container ?

I´d like to know if the previous version of your library for FM 16 works with container (in and outbound).
Thx

Add examples for new perform_script method

Current examples only show FMSv17 style of executing scripts.

TODO: Update examples.

Question: How do you create multiple records inside the portal? Portal inside the layout is a related record. Thanks! Please see the attached image for the reference

Displaying object in django template from portal

Hi, been this all day. How do you resolve this bug on your end related to portal looping in the template?

Provide access to the 'dataInfo' returned with each query

In the JSON returned by FileMaker is response.dataInfo which contains details about the query just performed. Of most interest to me at this time is the foundCount which gives the number of records matching the performed query.

Knowing this allows for the records to be loaded in batches which reduces the amount of data which is sent in any given payload from FileMaker.

How do you handle errors like you want to find a records but the record is not existing?

Delete multiple records

@davidhamann I love this fmrest. The only issue I'm having is deleting multiple records. You can't do:

df = server.get_records().to_df()
to_delete = df.loc[myfield.isnull(),'recordId']
###This next for loop will not work because recordId is updated on every 'POST'
for record_id in to_delete:
  server.delete_record(record_id)

But I think you can use delete request on multiple recordIds. I think that's possible because you can delete multiple records at once on the webd. I'm going to look at the API request on how WebD deletes multiple records tomorrow. But maybe you already know how to do this.

module 'fmrest' has no attribute 'Server'

I am having no issues when running on mac.

However, when I run on Ubuntu, I get:
AttributeError: module 'fmrest' has no attribute 'Server'

I've tested on two different fresh installs of Ubuntu on Raspberry Pi to no avail.

Question - How to access messages array when creating record

Is there a way to access the messages array (which contains "code" and "message") that's returned by the FileMaker Data API when a record is created please?

Thanks in advance.

[Question] Edit records within a portal

Hey there!

Thanks for this lovely package, I am doing testing to try update a FileMaker table with it, the steps I am following are:

1. Get desired record.
2. Loop through all the records within a portal in the record ( for rec in record[0]['portal_Other_Table'] )
3. Do stuff to rec and update it. (I am following this example to do so: https://github.com/davidhamann/python-fmrest/blob/master/examples/editing_data.ipynb)

But I am facing a fmrest.exceptions.FileMakerError: FileMaker Server returned error 101, Record is missing, I tried adding the data_sources=[{'database': 'Other_Table'}] to the FileMaker session (with and without username/password, as per https://github.com/davidhamann/python-fmrest/blob/master/examples/multiple_data_sources.ipynb) and it doesn't work.

I am new to FileMaker, so sorry if I am missing something obvious, so the question is:
Are we able to change records within portals with python-fmrest?

Example for accessing FMP database doesn't work...

import fmrest
fms = fmrest.Server( credentials, etc. )
Attribute Error: module 'fmrest' has no attribute 'Server'
dir(fmrest)
['builtins', 'cached', 'doc', 'file', 'loader', 'name', 'package', 'path', 'spec']

Please advise?

Timeout issue

Hi, how do I change the timeout constant in const.py, to something higher? Like 20 or 30. Im trying to generate PDFs with a perform_script, but some PDFs times out due to read_timeout = 10. I installed the wrapper via pip, but not sure how to use my own modified version of the wrapper.

New tag/release?

Would the new version (1.5.1) maybe justify a new tag / release?

Timeout when pulling a large set of records

When I try to pull the 2700x200 records from this layout I get a timeout error.
foundset = fms.get_records(range_=3000) foundset

returns

`---------------------------------------------------------------------------
RequestException Traceback (most recent call last)
in ()
----> 1 foundset = fms.get_records(range_=3000)
2 foundset

~/Documents/kai_ming/data_analysis/venv_km3/lib/python3.6/site-packages/fmrest/server.py in get_records(self, offset, range_, portals)
265 params['offset'] = offset
266 params['range'] = range_
--> 267 response = self._call_filemaker('GET', path, params=params)
268
269 return Foundset(self._process_foundset_response(response))

~/Documents/kai_ming/data_analysis/venv_km3/lib/python3.6/site-packages/fmrest/server.py in _call_filemaker(self, method, path, data, params)
412 data=data,
413 verify=self.verify_ssl,
--> 414 params=params
415 )
416

~/Documents/kai_ming/data_analysis/venv_km3/lib/python3.6/site-packages/fmrest/utils.py in request(*args, **kwargs)
26 return requests.request(*args, timeout=TIMEOUT, **kwargs)
27 except Exception as ex:
---> 28 raise RequestException(ex, args, kwargs) from None
29
30 def build_portal_params(portals, names_as_string=False):

RequestException: Request error: HTTPSConnectionPool(host='fake.address', port=443): Read timed out. (read timeout=10)`

Also, is there a better way to pull all records there than setting range to very large value you are sure is larger than the number of records?

Retrieve a list of all available databases hosted on FMS

It would be very nice to have the possibilty to query FMS for a list of all available databases, depending on being logged in or not and depending on the FMS preference for filtering databases set to 'on' or 'off'. See #46

Make http request timeout easily configurable

As per #2 (comment).

FileMakerError adds too much abstraction

Currently if FileMaker server returns an error an exception of type FileMakerError is raised. The same exception is used for all error types.

The problem that I am finding is that I now have to parse a lengthy text string to determine what the error actually was. In many instances this wouldn't be an issues, since it's likely to be fatal to the program execution and I'd need to log the exception (for example) and then provide some generic error response.

There are however times when you want to catch a specific error and behave differently for those. Immediate use cases I'm dealing with are

• no records (401)
• validation failed (503 - 507)

Would it be possible to either raise alternative exceptions for these errors, or expose the error code in the current exception?

botocore notauthorizedexception when deploying CloudServer on Heroku

Hi,

I think this is more of an issue with Cognito/Claris than this package, but wondering if you’ve got any suggestions for the issue where generating a token locally works fine, but when hosted leads to

botocore.errorfactory.notauthorizedexception: an error occurred (notauthorizedexception) when calling the respondtoauthchallenge operation: unable to login because of security reasons.

There’s a post about it on the community forums but doesn’t seem like there’s an actual solution

https://community.claris.com/en/s/question/0D53w00005SS3ijCAD/filemaker-cloud-api-unable-to-login-because-of-security-reasons-when-running-it-from-any-of-the-aws-servers

thanks!

Digital Signature

Good Day,

Is it possible to implement a digital signature and upload it as a container?

Server instance destructor error after connection error

Hi David,
Ian and I are including fmrest into the WB2 python project, and it's great to have it.
We are running into a bit of a problem when e.g. the admin forgot to enable the Data API on the server.
In that case fmrest returns an exception with server error 502, which we can trap.
But the destructor of the Server instance has a problem I think. It tries to log out. While it was never able to log in.
This generates another error that I don't know how to trap.

I'm just a noob in Python, but maybe it's a good idea to check in the destructor if the connection was successful.
Thanks, and keep up your great work!

FileMaker server version > 17 in README

The README says it requires filemaker server 17. I see from closed issues that it should work with version 19. If that's correct, would you be open to changing:

You need Python 3.6 and FileMaker Server/Cloud 17.

to

You need Python 3.6 or newer, and FileMaker Server/Cloud 17 or newer.

How to display results of find dynamically into a template?

How to display results of find dynamically into a template?

RequestException Max retires

Hi, First off I really appreciate the work you have done here. It has been a great tool for us.

I recently started randomly getting an error that I am not sure what to do with. The error started a week ago or so but resolved itself after making no changes to my code the next morning. I'm not even positive that fmrest is the source of the issue so if it is not please let me know. Here is the error I'm seeing when I make my request.

File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/fmrest/utils.py", line 29, in request
    raise RequestException(ex, args, kwargs) from None
fmrest.exceptions.RequestException: Request error: HTTPSConnectionPool(host='filemaker.example.domain', port=443): Max retries exceeded with url: /fmi/data/v1/databases/example/sessions/ (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x103b327b8>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))

note: some words were replaced for security.

please let me know if you have any insight or suggestions. Thanks.

Question: can I use python-fmrest to acceess data in a local .fmp12?

Hi David,
Your examples shows FileMaker Server Data API.

Can I also use python-fmrest to directly access a local Filemaker database(.fmp12) without Server Connection?
I want to extract some fields from a existing database to save in a CSV format.
Thanks

Let user chose version of Data API

Starting with FMS 19, the FileMaker Data API version requested can use 'v1', 'v2', or 'vLatest'.

Could we let the user chose the version and default to 'vLatest' in case the user does not supply any version?

Unless this breaks anything compared to the currently hard coded 'v1'...

SSL version issue on Windows

Hello! This may not be an issue w/ python-fmrest specifically but I get an error running on Windows Server 2016:

fmrest.exceptions.RequestException: Request error:
Max retries exceeded with url: /fmi/data/v1/databas
ON_NUMBER] wrong version number (_ssl.c:1123)')))

Same code works fine on my Mac. I have tried replacing the URL to the FM server with multiple things - the FQDN, localhost, just the DNS name w/ out domain, etc. The Windows machine I am running the Python script on is also the FM server.

Thanks!

Similar queries yield different foundset

Hello there!

Thanks again for this fantastic library.

I am experiencing a odd situation, two seemingly similar find() yield different data against a self-hosted instance:

quote_id = '<id-of-another-table>'

results1 = fm_session.find([
    {'_sk_Quote_ID': quote_id}
])

results2 = fm_session.find([
    {
        '_sk_Quote_ID': quote_id,
        'd__AnotherField': 'BEEP BOOP'
    }
])

Looping through the first foundset:

for record in results1:
    if record.d__AnotherField == 'BEEP BOOP':
        print(record.to_dict())

Nothing gets printed

Then printing the results of the second foundset outputs what I expected in the previous loop.

for record in results2:
    print(record.to_dict())

I also tried printing the d__AnotherField field and they are missing

for record in results1:
	print(record.d__AnotherField)

But none of the printed values match "BEEP BOOP" as if they weren't there.

Any idea why are those records are missing from the results1?

I can see though the webui/app that those are there, plus the second query also returns what's expected.

Handling of expired tokens

The backend application I'm building does an initial poll for data on startup and will then monitor other input sources for data which then require updates to be made to FileMaker.

It's entirely conceivable that it could be greater than the token timeout between input. This means I'm faced with a few choices:

• login, perform my action, and logout everywhere,
• trap for 952 everywhere, attempt a re-login, and try my query again,
• create a layer above this library which all calls are passed through which can handle the 952 error and do the re-login.

In abstractions which I've built on top of the DataAPI (in PHP and FileMaker) I've always had my 'library' handle the token (including expiration) for me.

Is this something which you would consider adding to this library? (If I sent a PR?) Or do you have another method for dealing with this I haven't thought of which is perhaps more graceful and less replication?

Building with PyInstaller

This isn't so much an issue as an FYI which could be added to the documentation to help future users.

I have need to build a Windows executable which makes use of python-fmrest (thanks :-. I'm using PyInstaller to do this for me. All goes well with the build but when attempting to run the resulting executable the error pkg_resources.DistributionNotFound: The 'python-fmrest' distribution was not found and is required by the application is received.

For reasons which aren't entirely clear to me PyInstaller isn't able to locate the python-rest metadata and so although it knows the module is required it doesn't correctly bundle it into the built application.

To resolve this create a new folder in your project folder (e.g. hooks). In there create a file hook-fmrest.py containing

from PyInstaller.utils.hooks import copy_metadata
datas = copy_metadata('python-fmrest')

Update your .spec file to include hookspath=['.\\hooks'], in the Analysis section at the top of the file (hookspath will most likely be there, but empty).

If you don't have a .spec file already then when first calling pyinstaller add --additional-hooks-dir=C:\\Path\\To\\Your\\hooks to the command called.

Hopefully this saves future users a couple of hours of head scratching!

Issue: Connection Pool

There was an error connecting to the server, the request timed out#012#012Request error: HTTPSConnectionPool(host='est-dev.chicodc.com', port=443): Max retries exceeded with url: /fmi/data/vLatest/databases/HRYCG%20SOLUTION%20LIVE%202020/sessions/ (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f2694aa0100>, 'Connection to est-dev.chicodc.com timed out. (connect timeout=60)'))

How do you handle this error if the FM database is more than 50k records?

edit_record() returning Filemaker Error 101, though the record is accessible by using find()

I am having trouble updating a records.

For almost all record numbers this code cooperates but occasionally it yields a "Filemaker Error Code 101 - Record Is Missing":

server = fmrest.Server(url=self.url,
                               user=user_name,
                               password=some_password,
                               database='UCPPC',
                               layout='rfilog_raw_table',
                               api_version='v1')
server.login()
server.edit_record(record_id=20818, field_data={'Answer': 'blah blah blah', 'Answer Date': '11/07/2021'})

...despite the fact that doing this gets the records object as expected:

server = fmrest.Server(url=self.url,
                               user=user_name,
                               password=some_password,
                               database='UCPPC',
                               layout='rfilog_raw_table',
                               api_version='v1')
server.login()
rec = server.get(20818)

...but then I am unaware of how one might update the record object for fields/attributes that cannot be attributes (eg 'Answer Date' field.)

This is for Filemaker Server 19.
The python code is Python 3.9.11 on Windows 10.
python-fmrest Ver 1.6.0

Any thoughts on resolving this would be appreciated.

Compatibility with PyInstaller - Add hook file.

PyInstaller gets confused when trying to generate an executable for my project because of the python-fmrest package. It doesn't include the python-fmrest metadata, likely because the top-level import module name is different ("fmrest" instead of "python-fmrest").

The fix for this is to include a pyuinstaller hook to tell pyinstaller to copy the package's metadata. I"ve made my own for my project, but it's best if it's included in your source.

A single file with the name "hook-fmrest.py" with the following content will do the trick.

from PyInstaller.utils.hooks import copy_metadata
datas = copy_metadata('python-fmrest')

Then you need to include information in "setup.cfg" to point pyinstaller to the hook file. See pyinstaller's docs for reference:
https://pyinstaller.readthedocs.io/en/stable/hooks.html#provide-hooks-with-package

Adding support for asyncio

I like python-fmrest, but it doesn't support asyncio.

I would like to make API calls like this.

async def f():
    await fms.login()
    record = await fms.get_record(1)
    return record

can't create new record if no records in table

when I attempt to create a record (fms.create_record) in a table with no current records, I get the following error:

Traceback (most recent call last):
  File "fmtest.py", line 51, in <module>
    records = fms.get_records()
  File "/usr/local/lib/python3.6/site-packages/fmrest/server.py", line 381, in get_records
    response = self._call_filemaker('GET', path, params=params)
  File "/usr/local/lib/python3.6/site-packages/fmrest/server.py", line 585, in _call_filemaker
    fms_messages[0].get('message', 'Unkown error'))
fmrest.exceptions.FileMakerError: FileMaker Server returned error 101, Record is missing

If I manually create a blank record in FileMaker Pro the python script works as intended, but with a blank record in the table.

FileMaker 19 compatible?

Is fmrest compatible with File Maker Server 19?
When I call login() I get:
"fmrest.exceptions.BadJSON: Expecting value: line 1 column 1 (char 0), 200 http response, content-type: text/html;charset=utf-8"
I am using fmrest 1.0.2.

pagination

Maybe its something obvious and I'm missing it (forgive me if that is the case) but do you have any examples of how I would go about looping through or handling additional pages of records. The get records function jumps straight into the data['fieldData'] being returned which is awesome but what if I want to be able to pull info out of the response['dataInfo'] such as the 'totalRecordCount' and 'foundCount' etc.

Thanks again for all you do.

raise RecordError('Not a valid record instance. record_id is missing.') from None

Hi - first ever GitHub error report. Apologies if I'm missing any key info.

Installing fmREST on a Raspberry Pi and getting the following error at install, and when I just try to make a test connection to a FM17 database:

"raise RecordError('Not a valid record instance. record_id is missing.') from None"

Full install error:

Downloading/unpacking python-fmrest
  Downloading python-fmrest-1.0.2.tar.gz
  Running setup.py (path:/tmp/pip-build-IDzUuN/python-fmrest/setup.py) egg_info for package python-fmrest
    /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'long_description_content_type'
      warnings.warn(msg)

Requirement already satisfied (use --upgrade to upgrade): requests in /usr/lib/python2.7/dist-packages (from python-fmrest)
Installing collected packages: python-fmrest
  Running setup.py install for python-fmrest
    /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'long_description_content_type'
      warnings.warn(msg)

      File "/usr/local/lib/python2.7/dist-packages/fmrest/server.py", line 241
        raise RecordError('Not a valid record instance. record_id is missing.') from None
                                                                                   ^
    SyntaxError: invalid syntax

      File "/usr/local/lib/python2.7/dist-packages/fmrest/record.py", line 75
        raise AttributeError(ex) from None
                                    ^
    SyntaxError: invalid syntax

      File "/usr/local/lib/python2.7/dist-packages/fmrest/utils.py", line 28
        raise RequestException(ex, args, kwargs) from None
                                                    ^
    SyntaxError: invalid syntax

      File "/usr/local/lib/python2.7/dist-packages/fmrest/foundset.py", line 85
        ) from ex
             ^
    SyntaxError: invalid syntax

Successfully installed python-fmrest
Cleaning up...

OS:
Linux pi-signage1 4.9.40-v7+ #1022 SMP Sun Jul 30 11:16:10 BST 2017 armv7l GNU/Linux

Initializer doesn't call super(), breaking MRO

Server().init appears to have no call to super(), which breaks MRO when creating a class user Server() as one of several baseclasses (multiple inheritance). This can break other class's code because it fails to call init() on classes that come after Server().init() in the MRO. Please add a super() call to all class initializers that can be sub-classed.

How to specify layout name?

It seems I can specify layout name while login only. How can I change it later to query different data from different layouts?

pip install error

Hi
I tried to install using pip3 install https://github.com/davidhamann/python-fmrest/archive/master.zip, but got the error below. Any suggestions? Thanks

Collecting https://github.com/davidhamann/python-fmrest/archive/master.zip
Downloading https://github.com/davidhamann/python-fmrest/archive/master.zip
- 1.2MB 1.3MB/s
ERROR: Complete output from command python setup.py egg_info:
ERROR: Traceback (most recent call last):
File "", line 1, in
File "C:\Users\CHRIST1\AppData\Local\Temp\pip-req-build-t_wnnsln\setup.py", line 4, in
long_description = ld.read()
File "c:\users\christian lan\appdata\local\programs\python\python36\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 721: character maps to
----------------------------------------
ERROR: Command "python setup.py egg_info" failed with error code 1 in C:\Users\CHRIST1\AppData\Local\Temp\pip-req-build-t_wnnsln\

Help with class Server login

Hello, I'm getting this error when trying to login using the Server class:

fmrest.exceptions.FileMakerError: FileMaker Server returned error 212, Authorization header 'Basic' invalid.

Any help or guidance on this would be great!

Present field reported as missing

I think I'm running into a bug where a field that is present in the layout is not found when doing a query.

foundset = fms.get_records(range_=2)
print(len(foundset[0].keys()))
print(foundset[0]['k.year'])

returns

206
15-16

but

find_query = [{'k.year': '15-16'}]
foundset = fms.find(find_query)

returns

---------------------------------------------------------------------------
FileMakerError                            Traceback (most recent call last)
<ipython-input-29-feafd4c0fd59> in <module>()
      1 find_query = [{'k.year': '15-16'}]
----> 2 foundset = fms.find(find_query)

~/Documents/kai_ming/data_analysis/venv_km3/lib/python3.6/site-packages/fmrest/server.py in find(self, query, sort, offset, range_, portals)
    311         }
    312 
--> 313         response = self._call_filemaker('POST', path, data=data)
    314 
    315         return Foundset(self._process_foundset_response(response))

~/Documents/kai_ming/data_analysis/venv_km3/lib/python3.6/site-packages/fmrest/server.py in _call_filemaker(self, method, path, data, params)
    423         if self.last_error != 0:
    424             raise FileMakerError(self._last_fm_error,
--> 425                                  response_data.get('errorMessage', 'Unkown error'))
    426 
    427         return response

FileMakerError: FileMaker Server returned error 102, Field is missing