sirfoga/pygce

:snake: A tool to export, save, and analyze your Garmin Connect data

pygce's Issues

Missing docs on what a chromedriver is and what to specify

Expected Behavior

Documentation should include what a chromedriver is and what to specify for that option, and should have a default if possible like the url does.

Actual Behavior

Neither the readme nor the -h output indicates what goes in this field.

Steps to Reproduce the Problem

1. Try to run it without specifying a chromedriver
2. Try to run it with -h
3. Try to read the readme file

Specifications

• pygce version (check pip3 show pygce): 2ca3adc
• Platform (e.g Windows 8, Mac OSX Sierra, Ubuntu 18.04 ...): Ubutun 16.04
• Url of connection (e.g https://connect.garmin.com, if you don't know it,
  then it's the stock one: https://connect.garmin.com): default
• Python version (check python --version): 2.7.12/3.5.2

Capture 15-min bin steps

Expected Behavior

Cmd line option should allow download 15-min bin steps

Actual Behavior

No such option

Details

e.g this page shows such data:

...
{"startGMT":"2018-12-13T23:00:00.0","endGMT":"2018-12-13T23:15:00.0","steps":0,"primaryActivityLevel":"sedentary","activityLevelConstant":true},{"startGMT":"2018-12-13T23:15:00.0","endGMT":"2018-12-13T23:30:00.0","steps":0,"primaryActivityLevel":"sedentary","activityLevelConstant":true},{"startGMT":"2018-12-13T23:30:00.0","endGMT":"2018-12-13T23:45:00.0","steps":0,"primaryActivityLevel":"sedentary","activityLevelConstant":true},{"startGMT":"2018-12-13T23:45:00.0","endGMT":"2018-12-14T00:00:00.0","steps":0,"primaryActivityLevel":"sedentary","activityLevelConstant":true},{"startGMT":"2018-12-14T00:00:00.0","endGMT":"2018-12-14T00:15:00.0","steps":0,"primaryActivityLevel":"sedentary","activityLevelConstant":true},{"startGMT":"2018-12-14T00:15:00.0","endGMT":"2018-12-14T00:30:00.0","steps":0,"primaryActivityLevel":"sedentary","activityLevelConstant":true},{"startGMT":"2018-12-14T00:30:00.0","endGMT":"2018-12-14T00:45:00.0","steps":0,"primaryActivityLevel":"sedentary","activityLevelConstant":true},{"startGMT":"2018-12-14T00:45:00.0","endGMT":"2018-12-14T01:00:00.0","steps":0,"primaryActivityLevel":"sedentary","activityLevelConstant":true},{"startGMT":"2018-12-14T01:00:00.0","endGMT":"2018-12-14T01:15:00.0","steps":0,"primaryActivityLevel":"sedentary","activityLevelConstant":false},{"startGMT":"2018-12-14T01:15:00.0","endGMT":"2018-12-14T01:30:00.0","steps":0,"primaryActivityLevel":"sleeping","activityLevelConstant":true},{"startGMT":"2018-12-14T01:30:00.0","endGMT":"2018-12-14T01:45:00.0","steps":0,"primaryActivityLevel":"sleeping","activityLevelConstant":true},{"startGMT":"2018-12-14T01:45:00.0","endGMT":"2018-12-14T02:00:00.0","steps":0,"primaryActivityLevel":"sleeping","activityLevelConstant":true},{"startGMT":"2018-12-14T02:00:00.0","endGMT":"2018-12-14T02:15:00.0","steps":0,"primaryActivityLevel":"sleeping","activityLevelConstant":true},{"startGMT":"2018-12-14T02:15:00.0","endGMT":"2018-12-14T02:30:00.0","steps":0,"primaryActivityLevel":"sleeping","activityLevelConstant":true},{"startGMT":"2018-12-14T02:30:00.0","endGMT":"2018-12-14T02:45:00.0","steps":0,"primaryActivityLevel":"sleeping","activityLevelConstant":true},{"startGMT":"2018-12-14T02:45:00.0","endGMT":"2018-12-14T03:00:00.0","steps":0,"primaryActivityLevel":"sleeping","activityLevelConstant":true},{"startGMT":"2018-12-14T03:00:00.0","endGMT":"2018-12-14T03:15:00.0","steps":0,"primaryActivityLevel":"sleeping","activityLevelConstant":true},{"startGMT":"2018-12-14T03:15:00.0","endGMT":"2018-12-14T03:30:00.0","steps":0,"primaryActivityLevel":"sleeping","activityLevelConstant":true}
...

Specifications

• pygce version (check pip3 show pygce): 2.0.6
• Platform (e.g Windows 8, Mac OSX Sierra, Ubuntu 18.04 ...): any
• Url of connection (e.g https://connect.garmin.com, if you don't know it,
  then it's the stock one: https://connect.garmin.com):
• Python version (check python --version): 3.6

how to use this script for a different garmin website

Hi, I find the script is made for https://connect.garmin.com, can this script also work for https://connect.garmin.cn?

Thank you for your time

'NoneType' object has no attribute 'date'

Hi,
So 1st, I wanted to say a Massive Thankyou.
I've been pestering the Garmin people for API and they turn you away saying - Pay the Money, or Jog on!

So - I have got Virtualbox, and installed Ubuntu LTS 18.04.1.

I have got Python 3.6 on Ubuntu - This satisfies some of the min installation critera of Numpy,
https://stackoverflow.com/questions/42662104/how-to-install-pip-for-python-3-6-on-ubuntu-16-10

Anyway - I have run the Setup for pygce as you have suggested in your detailed Readme.
I have given the correct parameters, to the pygce application.

It starts Chromium in Automation mode. It logs in successfully as me.
It goes to "https://connect.garmin.com/modern/" and then stops with the following exception.

Getting day 2018-10-02 00:00:00
Getting user dashboard from https://connect.garmin.com/modern/
Message:
Traceback (most recent call last):
File "/usr/local/bin/pygce", line 11, in
load_entry_point('pygce==1.9.8', 'console_scripts', 'pygce')()
File "/home/eastabrooka/.local/lib/python3.6/site-packages/pygce/cli.py", line 133, in main
bot.save_json_days(days[0], days[1], path_out)
File "/home/eastabrooka/.local/lib/python3.6/site-packages/pygce/models/bot.py", line 227, in save_json_days
print("Parsing day", str(d.date))
AttributeError: 'NoneType' object has no attribute 'date'

Expected Behavior

The pygce to pull the daily-summary for yesterday

Actual Behavior

Exception.

Steps to Reproduce the Problem

1. pygce -user [email protected] -p password -c /home/eastabrooka/Downloads/chromedriver -d 2018-10-02 -f json -o ./output.txt
   1.exception occurs.

Specifications

• pygce version (check pip3 show pygce):
  Name: pygce
  Version: 1.9.8
  Summary: pygce is an unofficial Garmin Connect data exporter.
  Home-page: https://github.com/sirfoga/pygce
  Author: sirfoga
  Author-email: [email protected]
  License: UNKNOWN
  Location: /home/eastabrooka/.local/lib/python3.6/site-packages
  Requires: bs4, pyhal, lxml, numpy, sklearn

• Platform (e.g Windows 8, Mac OSX Sierra, Ubuntu 18.04 ...):
  Ubuntu LTS 18.04.1.

• Url of connection (e.g https://connect.garmin.com):
  Stock - Whatever Pygce uses as default

• Python version (check python --version):

eastabrooka@GarminScraper:~/pygce$ python3 --version
Python 3.6.6

Thank you again for posting Pygce, its very useful !

Could not find suitable distribution for Requirement.parse('pyhal')

Hi.!

I try to install pygce. After running **python3 setup.py install --force ** i got following Error

Installed /usr/lib/python3.4/site-packages/pygce-0.1.2-py3.4.egg Processing dependencies for pygce==0.1.2 Searching for pyhal Reading https://pypi.python.org/simple/pyhal/ No local packages or download links found for pyhal error: Could not find suitable distribution for Requirement.parse('pyhal')

System: openSUSE Leap 42.2

BYe
Harald

Can i save gpx files from garmin connect, too?!?

Only a question: Can i save gpx files from garmin connect, too?!?

No module named 'hal.internet.selenium_bots'

Expected Behavior

"pygce -h" returns a help screen

Actual Behavior

"pygce -h" returns the following:

ModuleNotFoundError: No module named 'hal.internet.selenium_bots'

Traceback (most recent call last):
File "/Users/paredes/venv.pygce/bin/pygce", line 7, in
from pygce.cli import main
File "/Users/paredes/venv.pygce/lib/python3.7/site-packages/pygce/cli.py", line 9, in
from pygce.models.bot import GarminConnectBot
File "/Users/paredes/venv.pygce/lib/python3.7/site-packages/pygce/models/bot.py", line 10, in
from hal.internet.selenium_bots import SeleniumForm
ModuleNotFoundError: No module named 'hal.internet.selenium_bots'
(venv.pygce)

Steps to Solve the Problem

1. Reinstalled Python
2. Reinstalled Selenium
3. Reinstalled chromedriver
4. Reinstalled pygce

Specifications

• pygce version (check pip3 show pygce):
  1.9.8
• Platform (e.g Windows 8, Mac OSX Sierra, Ubuntu 18.04 ...):
  Mac OSX High Sierra ver 10.13.6
• Url of connection (e.g https://connect.garmin.com, if you don't know it,
  then it's the stock one: https://connect.garmin.com):
  Https://connect.garmin.com/en-US/signin
• Python version (check python --version):
  Python 3.7.0
  -Selenium ver 3.141.0; urllib3 ver 1.24.1

'WebDriver' object has no attribute 'fill_form_field'

Expected Behavior

Scrape data from Garmin Connect.

Actual Behavior

It gets stuck on the login screen. The exact error is as follows:

2020-07-29 20:52:29,376 - pygce - **ERROR - 'WebDriver' object has no attribute 'fill_form_field'**
2020-07-29 20:52:29,376 - pygce - DEBUG - GET https://connect.garmin.com/modern/
2020-07-29 20:52:32,031 - pygce - DEBUG - attempt 0

So it looks as though it has something to do with the pyhal package? I was getting a different error before, but changed the imports to:

from hal.internet.selenium.forms import SeleniumFormFiller

which fixed that issue. And now I am getting the issue that 'WebDriver' object has no attribute 'fill_form_field'.

Steps to Reproduce the Problem

1. pygce -user [email protected] -p redacted -c /c/Users/name/documents/folder/fitness/scrapesteps/chromedriver.exe -d 2018-09-05 2020-07-28 -o /c/Users/name/documents/folder/fitness/scrapesteps -f json

Specifications

• pygce version (check pip3 show pygce): 1.9.8
• Platform (e.g Windows 8, Mac OSX Sierra, Ubuntu 18.04 ...): Windows 10
• Url of connection (e.g https://connect.garmin.com, if you don't know it,
  then it's the stock one: https://connect.garmin.com): Default connection.
• Python version (check python --version): 3.8.3

Usage instructions indicate -f option to specify format, but using that option results in an error

Expected Behavior

Adding "-f json" to execution should specify that the output format of the file should be json.

Actual Behavior

Specifying "-f json" results in an error:

unrecognized arguments: -f json

Steps to Reproduce the Problem

1. Run pygce -h
2. Read the usage summary: "-f <format of output file [json, csv]>"
3. Attempt to use the flag mentioned

Specifications

• pygce version (check pip3 show pygce):2.1.2
• Platform (e.g Windows 8, Mac OSX Sierra, Ubuntu 18.04 ...):ubuntu 16.04
• Url of connection (e.g https://connect.garmin.com, if you don't know it,
  then it's the stock one: https://connect.garmin.com): default
• Python version (check python --version): 2.7.12/3.5.2

Pygce crashes with "No such file or directory" error

Expected Behavior

pygce should run and produce an output file, or a reasonable error should be shown so the user can take corrective action.

Actual Behavior

Program crashes with backtrace and error. No indication is provided as to what directory is missing or why the application expects it to be there, nor what the user can do to fix it.

Steps to Reproduce the Problem

1.pygce -user (redacted email address) -c /path/to//Downloads/chromedriver/chromedriver -o foo.json -d 2019-07-01 -p (redacted password)

1. Observe crash output:

Traceback (most recent call last):
File "/usr/local/bin/pygce", line 9, in
load_entry_point('pygce==2.1.2', 'console_scripts', 'pygce')()
File "/usr/local/lib/python3.5/dist-packages/pygce-2.1.2-py3.5.egg/pygce/cli.py", line 124, in main
File "/usr/local/lib/python3.5/dist-packages/pygce-2.1.2-py3.5.egg/pygce/cli.py", line 115, in check_args
File "/usr/lib/python3.5/os.py", line 241, in makedirs
mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: ''

Specifications

• pygce version (check pip3 show pygce): 2.1.2
• Platform (e.g Windows 8, Mac OSX Sierra, Ubuntu 18.04 ...): ubuntu 16.04
• Url of connection (e.g https://connect.garmin.com, if you don't know it,
  then it's the stock one: https://connect.garmin.com): default
• Python version (check python --version): 2.7.12/3.5.2

daily step format

Expected Behavior

Is it possible to output the 15 min bin step counts without formatting such that "1.897" is outputted as "1897" or "835.0" is outputted as "835" ?

Actual Behavior

"1897" steps is output as "1897"
"835" is output as "835.0"

Steps to Reproduce the Problem

1. login
2. download daily steps details

Specifications

• pygce version (check pip3 show pygce): 2.0.6
• Platform (e.g Windows 8, Mac OSX Sierra, Ubuntu 18.04 ...): Ubuntu 18.04
• Url of connection (e.g https://connect.garmin.com, if you don't know it,
  then it's the stock one: https://connect.garmin.com):
• Python version (check python --version): 3.6.7

Garmin step details output off by varying amounts

Expected Behavior

Garmin step details output for the day

Actual Behavior

However, the 15 min bin counts are off by varying amounts. When comparing the pygce json generated 15 min counts against what is displayed on the Garmin step details screen, I see that the generated json files are always missing the last 4 or 5 hours of data.

The 15 min bin output for day 2018-05-10:

{"15-min bins": [{"time": "2018-05-10 04:00:00", "steps": 0}, {"time": "2018-05-10 04:15:00", "steps": 0}, . . .etc etc etc}

Steps to Reproduce the Problem

1. login
2. view step details for a day

Specifications

• pygce version (check pip3 show pygce): 2.0.6
• Platform (e.g Windows 8, Mac OSX Sierra, Ubuntu 18.04 ...): Ubuntu 18.04
• Url of connection (e.g https://connect.garmin.com, if you don't know it,
  then it's the stock one: https://connect.garmin.com):
• Python version (check python --version): 3.6.7

module not found...

Hi! I get the following on my Raspi when running pygce:

pi@raspberrypi:~/pygce $ pygce
Traceback (most recent call last):
  File "/usr/local/bin/pygce", line 11, in <module>
    load_entry_point('pygce==2.1.2', 'console_scripts', 'pygce')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 561, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2631, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2291, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2297, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python3.5/dist-packages/pygce-2.1.2-py3.5.egg/pygce/cli.py", line 9, in <module>
  File "/usr/local/lib/python3.5/dist-packages/pygce-2.1.2-py3.5.egg/pygce/models/bot.py", line 19, in <module>
ImportError: No module named 'models'

Any idea what it might be and how to fix it?

Cheers,
Jens

Example usage shows -u username, but using -u username results in an error

Expected Behavior

The output from -h "usage: -u <username (email) to login to Garmin Connect>" should match how arguments are actually parsed.

Actual Behavior

-h says to use -u to specify the username, but actually running pygce with -u username fails because -u is ambiguous and could mean -url.

Steps to Reproduce the Problem

1. Run pygce -h and observe the first line of output
2. Attempt to run pygce according to the "usage" line, using -u
3. Observe that the application fails to run, stating that -u is ambiguous.

Specifications

• pygce version (check pip3 show pygce): cd3ebec
• Platform (e.g Windows 8, Mac OSX Sierra, Ubuntu 18.04 ...): ubuntu 16.04
• Url of connection (e.g https://connect.garmin.com, if you don't know it,
  then it's the stock one: https://connect.garmin.com): default
• Python version (check python --version): 3.5.2

Related issue: #13.

Daily summary content failing to load

Expected Behavior

Daily summary content successfully loads in chromedriver and is able to be scraped.

Actual Behavior

Daily summary content does not load (menu bars do), as shown in screenshot below. This page will load in a normal browsing tab (ie this same URL has been copied across to a normal chrome window and loads instantly).

Note that the timeout length has been extended to 60s and the same issue still occurs.

Steps to Reproduce the Problem

1. Install pygce
2. Run with single date.
3. Wait for page to not load.

Specifications

• pygce version (check pip3 show pygce):
  Version: 2.1.2

• Platform (e.g Windows 8, Mac OSX Sierra, Ubuntu 18.04 ...):
  macOS Catalina (10.15.4)

• Url of connection (e.g https://connect.garmin.com, if you don't know it,
  then it's the stock one: https://connect.garmin.com):
  "https://connect.garmin.com"

• Python version (check python --version):
  Python 3.8.3

cannot import SeleniumFormFiller

Steps to Reproduce the Problem

1. git clone the repository
2. install with given command "python3 setup.py install --force"
3. run pygce -h which gives following output

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/bin/pygce", line 11, in
load_entry_point('pygce==2.0.6', 'console_scripts', 'pygce')()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/init.py", line 487, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/init.py", line 2728, in load_entry_point
return ep.load()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/init.py", line 2346, in load
return self.resolve()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/init.py", line 2352, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pygce-2.0.6-py3.7.egg/pygce/cli.py", line 9, in
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pygce-2.0.6-py3.7.egg/pygce/models/bot.py", line 10, in
ImportError: cannot import name 'SeleniumFormFiller' from 'hal.internet.selenium' (/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/hal/internet/selenium/init.py)

Solution on my machine

change one line in bot.py
from:
from hal.internet.selenium import SeleniumFormFiller
to:
from hal.internet.selenium.forms import SeleniumFormFiller

not sure if this is a quirk on my machine of more "widespread" as im not that familiar with python yet