freeseek/getmyancestors

I do not update this software anymore. Linekio is currently maintaining the new version.

getmyancestors's Issues

Needs updating to use OAuth2

Really like the look of this! I hope you're still intending on maintaining it - unbelievably there seems to be no (other) lightweight way of exporting your tree.

Bizarre that FS don't give this option themselves. I could understand if it was "we don't want you using other tools" - but then they let third-party software do it.. weird.

Anyway, if you're no longer interested I may get around to it myself... But I saw you link this repo in a couple places saying you'd be up for developing it if there's interest, so I guess I'm hoping that's still true :)

DATE format

Test command:
./getmyancestors.py -a 0 -d 0 -u xxxxx -p xxxxx -i KWZG-8XF -v -o jn.ged

Sometimes the DATEs of certain people in my GEDCOM files look like this:
1 BIRT
2 DATE 18770918

(This causes the birth/death dates to appear malformed when I import the GEDCOM into Ancestry.com)

Compared with other GEDCOM files I've seen, the DATE should look something like this:
1 BIRT
2 DATE 18 September 1877

Although I can write a script or use commandline tools (date -f - +'%d %B %Y', etc) to correct these dates in the resulting GEDCOM, (or better yet, revise the DATEs in FamilySearch for everyone in my tree), it would be easiest to change getmyancestors.py to output the date in "long format" like above.
How to do it? Not sure.. Maybe you could use strftime()?

Output file is empty when running with -a 99 -d 99

If I run getmyancestors with only the -o output file parameter, it downloads 4 generations and generates the file correctly. If I run it with -a 99 -d 99 -v, it fetches a bunch of records and then exits, but the output file is empty, and there is no error message or warning reported.

TypeError: must be str, not NoneType

Before saving the file, getmyancestor throws the following exception:

Traceback (most recent call last):
  File "getmyancestors.py", line 943, in <module>
    tree.print(args.o)
  File "getmyancestors.py", line 827, in print
    self.indi[fid].print(file)
  File "getmyancestors.py", line 600, in print
    file.write('1 OCCU ' + o.value + '\n')
TypeError: must be str, not NoneType

May have been caused by a 204 return code from FamilyTree.

TypeError: Can't convert 'NoneType' object to str implicitly

Thanks for this tool! I'm trying to download my tree and it runs for a while and downloads maybe 100 or more JSON files, then crashes with:

$ ./getmyancestors.py -a 7 -m -o /home/hans/Downloads/familysearch.gedcom -v
...snip...
[2018-01-01 12:43:21]: Downloading: https://familysearch.org/platform/tree/couple-relationships/MZYH-F8B/notes.json
[2018-01-01 12:43:23]: Status code: 204
[2018-01-01 12:43:23]: Status code: 204
[2018-01-01 12:43:23]: Status code: 204
[2018-01-01 12:43:23]: Status code: 204
[2018-01-01 12:43:23]: Status code: 204
Traceback (most recent call last):
  File "./getmyancestors.py", line 943, in <module>
    tree.print(args.o)
  File "./getmyancestors.py", line 827, in print
    self.indi[fid].print(file)
  File "./getmyancestors.py", line 608, in print
    file.write('1 _MILT ' + o.value + '\n')
TypeError: Can't convert 'NoneType' object to str implicitly

python 3.4 vs 3.5

The code says:

sys.stderr.write('Python >= 3.4 is required to run this script\n')

yet python3.4 says:

  File "./getmyancestors.py", line 891
    async def download_tree(func, todo, n, loop):
            ^
SyntaxError: invalid syntax

I am running python 3.4....
Async def appears to be for python 3.5 only:

https://docs.python.org/3/reference/compound_stmts.html#coroutines

Direct Ancestors Request

Good Day,
This might be a pipe dream, but is it possible to only download Direct Ancestors?
Thank you,
Torrey Mathie

503 HTTPError

Seems that Family Tree has been updated. After they made some changes to their site, getmyancestors.py returns the following:

./getmyancestors.py -a 2 -d 0 -i L8JP-GLC -v -o 2gen.ged
Enter FamilySearch username: mhs1
Enter FamilySearch password: 
[2016-07-04 03:04:59]: Downloading: https://familysearch.org/auth/familysearch/login
[2016-07-04 03:04:59]: FamilySearch developer key: 3Z3L-Z4GK-J7ZS-YT3Z-Q4KY-YN66-ZX5K-176R
[2016-07-04 03:04:59]: Downloading: https://api.familysearch.org/identity/v2/login
[2016-07-04 03:04:59]: Status code: 503
[2016-07-04 03:04:59]: HTTPError

File "<stdin>", line 1 error

Python 3.4.3 w/requests module. Here's my error...

root@d:/home/a/Downloads/getmyancestors-master# python3.4
Python 3.4.3 (default, Mar 23 2015, 12:32:24)
[GCC 4.7.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> getmyancestors.py -a 4 -d 0 -u nickflyer -p *** -i LVF2-FVB -o out.ged
  File "<stdin>", line 1
    getmyancestors.py -a 4 -d 0 -u nickflyer -p *** -i LVF2-FVB -o out.ged
                         ^
SyntaxError: invalid syntax
>>>

Line 39: # login (https://familysearch.org/developers/docs/guides/oauth1/login)
On that page, is says "Warning: OAuth 1.0 has been deprecated in favor of OAuth 2.0."
Maybe that's why?

Failing on one specific person

I've isolated one specific person that is causing getmyancestors to fail.. Wonder why we are failing on this specific person.. I can build GEDCOMs from any other family. Could there be something wrong with the individual's data in FamilySearch?

Test command:

mhs@ny1:~$ ./getmyancestors.py -a 0 -d 0 -u xxxx -p xxxx -i LCFZ-DXV -v -o rw.ged
Downloading: https://familysearch.org/platform/tree/persons/LCFZ-DXV.json
Status code: 200
Traceback (most recent call last):
File "./getmyancestors.py", line 392, in
tree.indi[fid] = Indi(fid)
File "./getmyancestors.py", line 136, in init
self.buriplac = y['place']['original'] if 'place' in y else None
KeyError: 'original'

Max retries exceeded... Caused by NewConnectionError

When I try pulling large GEDs, 100% of the time I get different errors at random times... sometimes the log file is 50k and sometimes it's 4mb. Here's what my most recent error... maybe it just needs a resume function?

FamilySearch ERROR.txt

$ python3.4 getmyancestors.py -a 10 -d 10 -u ***** -p ***** -i LVF5-WPQ -o A.ged" -l a.log -v
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 135, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/usr/local/lib/python3.4/site-packages/requests/packages/urllib3/util/connection.py", line 66, in create_connection
    for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
  File "/usr/local/lib/python3.4/socket.py", line 530, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    body=body, headers=headers)
  File "/usr/local/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 345, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 782, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 215, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 144, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
requests.packages.urllib3.exceptions.NewConnectionError: <requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3d1ac987f0>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/requests/adapters.py", line 370, in send
    timeout=timeout
  File "/usr/local/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 609, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.4/site-packages/requests/packages/urllib3/util/retry.py", line 271, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
requests.packages.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='familysearch.org', port=443): Max retries exceeded with url: /platform/tree/persons/KCQ9-5C5.json (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3d1ac987f0>: Failed to establish a new connection: [Errno -2] Name or service not known',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ma/Dropbox/MaAndZee/getmyancestors/getmyancestors.py", line 57, in get_url
    r = requests.get(url, cookies = { 'fssessionid' : self.fssessionid })
  File "/usr/local/lib/python3.4/site-packages/requests/api.py", line 69, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python3.4/site-packages/requests/api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.4/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.4/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.4/site-packages/requests/adapters.py", line 423, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='familysearch.org', port=443): Max retries exceeded with url: /platform/tree/persons/KCQ9-5C5.json (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3d1ac987f0>: Failed to establish a new connection: [Errno -2] Name or service not known',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ma/Dropbox/MaAndZee/getmyancestors/getmyancestors.py", line 428, in <module>
    for child in get_family(fid):
  File "/home/ma/Dropbox/MaAndZee/getmyancestors/getmyancestors.py", line 346, in get_family
    process_trio(father,mother,child)
  File "/home/ma/Dropbox/MaAndZee/getmyancestors/getmyancestors.py", line 308, in process_trio
    tree.indi[child] = Indi(child)
  File "/home/ma/Dropbox/MaAndZee/getmyancestors/getmyancestors.py", line 113, in __init__
    data = fs.get_url(url)
  File "/home/ma/Dropbox/MaAndZee/getmyancestors/getmyancestors.py", line 68, in get_url
    if 'reason' in e.args[0]:
TypeError: argument of type 'MaxRetryError' is not iterable

No Issue - Positive feedback

Hello,
I just wanted to say that this tool is very handy. I a running 7 searches at the same time and have had no issues in the last 10. One has been running for over 46 hours, if i tried any of this in any other Genealogy software they would have all failed long ago. So, in short, Thank you

Future exception was never retrieved; Set changed size during iteration

I got this exception multiple times while trying to download with -a 99 -d 1

Future exception was never retrieved
future: <Future finished exception=RuntimeError('Set changed size during iteration',)>
Traceback (most recent call last):
  File "/usr/lib64/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "getmyancestors.py", line 765, in add_parents
    self.add_trio(father, mother, fid)
  File "getmyancestors.py", line 750, in add_trio
    self.add_indi(father)
  File "getmyancestors.py", line 741, in add_indi
    self.indi[fid] = Indi(fid, self)
  File "getmyancestors.py", line 442, in __init__
    self.sources.add((self.tree.add_source(json),))
  File "getmyancestors.py", line 804, in add_source
    for s in self.sources:
RuntimeError: Set changed size during iteration

individual is not respected

A gedcom file is successfull created, but the individual i entered in the pyhton script call is not respected. The gedcom file always containts the exact same content and individuals.

New login URL

Something changed with familysearch and causes a ValueError. Just change the login URL:
https://familysearch.org/auth/familysearch/login
by:
https://www.familysearch.org/auth/familysearch/login