avelkoski/frb

Python Client for Interfacing with the Federal Reserve Bank of St. Louis' Economic Data API (FRED®)

frb's Issues

Error in importing when multiple packages are named as fred

If another pakcage named fred (also mentioned by the FRED in API resources) is pip downloaded, python interpreter can't find path to module fred from FRB local repo.

It is necessary to uninstall both packages and download again FRB for Python to find again fred module from FRB.

Mapping between 'id' and 'series_id'

Is there a mapping between 'id' (an integer) and 'series_id' (a string) ? Some functions require one or the other. Thank you.

python 3, json object must be str, not 'bytes'

Have you used this with python 3? there seems to be a few json compatibility issues, the object has to be decoded to a string

Attribute Error when querying DGS10; 'Dataframe object has no attribute "convert_objects"'

I'm having an issue when I try to pull data from the 10 year yield treasure 'DGS10'. I'm using frb version 1.1.4

Below is the error messages:

Traceback (most recent call last):
File "main.py", line 106, in
main()
File "main.py", line 92, in main
series_10y = pull_10y_yield(dt_start, dt_stop)
File "main.py", line 65, in pull_10y_yield
df_10y = fred_conn.series.observations(
File "C:\Users\Frosty SB 02\Anaconda3\envs\emba_fsa\lib\site-packages\fred\utils_init.py", line 17, in _wrapped
return func(*args,params=params,**kwargs)
File "C:\Users\Frosty SB 02\Anaconda3\envs\emba_fsa\lib\site-packages\fred\clients\eseries.py", line 175, in observations
response = get_request(self.url_root,self.api_key,path,response_type,params)
File "C:\Users\Frosty SB 02\Anaconda3\envs\emba_fsa\lib\site-packages\fred\helpers_init.py", line 117, in _get_request
response = dispatch(response_type)(content)
File "C:\Users\Frosty SB 02\Anaconda3\envs\emba_fsa\lib\site-packages\fred\helpers_init.py", line 47, in _dict
response = data_frame(content).to_dict(orient='records')
File "C:\Users\Frosty SB 02\Anaconda3\envs\emba_fsa\lib\site-packages\fred\helpers_init.py", line 58, in _data_frame
final_frame = convert(frame)
File "C:\Users\Frosty SB 02\Anaconda3\envs\emba_fsa\lib\site-packages\fred\helpers_init.py", line 36, in convert
frame = frame.convert_objects(convert_numeric=True)
File "C:\Users\Frosty SB 02\Anaconda3\envs\emba_fsa\lib\site-packages\pandas\core\generic.py", line 5139, in getattr
return object.getattribute(self, name)
AttributeError: 'DataFrame' object has no attribute 'convert_objects'

Below is the applicable code:

fred_conn = Fred(
        api_key = 'my-api-key',
        response_type = 'df'
    )
    df_10y = fred_conn.series.observations(
        series_id = 'DGS10',
        observation_start = date_start.strftime('%Y-%m-%d'),
        observation_end = date_stop.strftime('%Y-%m-%d'),
        frequency = 'd'
    )

    print(df_10y[:10])

Below is my list of python packages used:

Name Version Build Channel
astroid 2.4.2 py38_0
blas 1.0 mkl
ca-certificates 2020.10.14 0
certifi 2020.6.20 py38_0
chardet 3.0.4 pypi_0 pypi
colorama 0.4.4 py_0
cycler 0.10.0 py38_0
frb 1.1.4 pypi_0 pypi
freetype 2.10.4 hd328e21_0
icu 58.2 ha925a31_3
idna 2.10 pypi_0 pypi
intel-openmp 2020.2 254
isort 5.6.4 py_0
jpeg 9b hb83a4c4_2
kiwisolver 1.2.0 py38h74a9793_0
lazy-object-proxy 1.4.3 py38he774522_0
libpng 1.6.37 h2a8f88b_0
libtiff 4.1.0 h56a325e_1
lxml 4.6.1 pypi_0 pypi
lz4-c 1.9.2 hf4a77e7_3
matplotlib 3.3.2 0
matplotlib-base 3.3.2 py38hba9282a_0
mccabe 0.6.1 py38_1
mkl 2020.2 256
mkl-service 2.3.0 py38hb782905_0
mkl_fft 1.2.0 py38h45dec08_0
mkl_random 1.1.1 py38h47e9c7a_0
numpy 1.19.1 py38h5510c5b_0
numpy-base 1.19.1 py38ha3acd2a_0
olefile 0.46 py_0
openssl 1.1.1h he774522_0
pandas 1.1.3 py38ha925a31_0
pandas-datareader 0.9.0 pypi_0 pypi
pillow 8.0.0 py38hca74424_0
pip 20.2.4 py38_0
pylint 2.6.0 py38_0
pyparsing 2.4.7 py_0
pyqt 5.9.2 py38ha925a31_4
python 3.8.5 h5fd99cc_1
python-dateutil 2.8.1 py_0
pytz 2020.1 py_0
qt 5.9.7 vc14h73c81de_0
requests 2.24.0 pypi_0 pypi
setuptools 50.3.0 py38h9490d1a_1
sip 4.19.13 py38ha925a31_0
six 1.15.0 py_0
sqlite 3.33.0 h2a8f88b_0
tk 8.6.10 he774522_0
toml 0.10.1 py_0
tornado 6.0.4 py38he774522_1
urllib3 1.25.11 pypi_0 pypi
vc 14.1 h0510ff6_4
vs2015_runtime 14.16.27012 hf0eaf9b_3
wheel 0.35.1 py_0
wincertstore 0.2 py38_0
wrapt 1.11.2 py38he774522_0
xz 5.2.5 h62dcd97_0
zlib 1.2.11 h62dcd97_4
zstd 1.4.5 h04227a9_0

Publish ssl_verify fix to PyPi

This patch:a79bbed#diff-e4e93ea5b92e07294d7b5d8df517d31a91db251d8824bc05ea1189db935aac9b was made since the last publication to PyPi. Please publish this fix! Thank you.

convert_objects error

When I run fr = Fred(api_key='xxx', response_type='df')
I get the following error:
AttributeError: 'DataFrame' object has no attribute 'convert_objects'

The program will run fine if the response type is xml or json but gives this error if i try to use any of the other types.

Enable use of proxies & document its usage

Hi @avelkoski,
can we document use of proxies and subsequent use of this python module ?
Or, if necessary, expose ProxyHandler ?

https://github.com/avelkoski/FRB/blob/master/fred/helpers/__init__.py#L141
https://github.com/avelkoski/FRB/blob/master/fred/__init__.py

What I am trying is

import os
import urllib
proxy = 'http://username:password@enterprise proxy/'

os.environ['http_proxy'] = proxy 
os.environ['HTTP_PROXY'] = proxy
os.environ['https_proxy'] = proxy
os.environ['HTTPS_PROXY'] = proxy

Then, which works well

import os
print(os.environ.get('http_proxy'))
print(os.environ.get('https_proxy'))

Then:

from fred import Fred
fred = Fred(api_key='xxx', response_type='dict') 
params = {'limit':10}
res = fred.release.tags(51,params=params)
print(res)

fails with <urlopen error [Errno 111] Connection refused>

However, R and Python with different use case but still with proxy do seem to work.

Stack trace when loading category 33845

fr = Fred(api_key=api_key,response_type='df')
fr.category.series(33845, params={'limit':1000, 'offset': 0})

---

TypeError Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/pandas/core/tools/datetimes.py in _convert_listlike(arg, box, format, name, tz)
376 try:
--> 377 values, tz = conversion.datetime_to_datetime64(arg)
378 return DatetimeIndex._simple_new(values, name=name, tz=tz)

pandas/_libs/tslibs/conversion.pyx in pandas._libs.tslibs.conversion.datetime_to_datetime64()

TypeError: Unrecognized value type: <class 'str'>

During handling of the above exception, another exception occurred:

OutOfBoundsDatetime Traceback (most recent call last)
/tmp/ipykernel_333/1514929282.py in
1 fr = get_fred_api()
2
----> 3 fr.category.series(33845, params={'limit':RESPONSE_LIMIT, 'offset': 0})

/usr/local/lib/python3.7/dist-packages/fred/utils/init.py in _wrapped(*args, **kwargs)
15 if p in kwargs:
16 params[p] = kwargs.pop(p)
---> 17 return func(*args,params=params,**kwargs)
18 return _wrapped
19 return _wrapper

/usr/local/lib/python3.7/dist-packages/fred/clients/categories.py in series(self, category_id, response_type, params)
93 response_type = response_type if response_type else self.response_type
94 if response_type != 'xml': params['file_type'] = 'json'
---> 95 response = _get_request(self.url_root,self.api_key,path,response_type,params)
96 return response
97

/usr/local/lib/python3.7/dist-packages/fred/helpers/init.py in _get_request(url_root, api_key, path, response_type, params)
115 url = _url_builder(url_root,api_key,path,params)
116 content = _fetch(url)
--> 117 response = _dispatch(response_type)(content)
118 return response

/usr/local/lib/python3.7/dist-packages/fred/helpers/init.py in _data_frame(content)
56 key = [x for x in response.keys() if x in c.response_data][0]
57 frame = DataFrame(response[key])
---> 58 final_frame = _convert(frame)
59 return final_frame
60

/usr/local/lib/python3.7/dist-packages/fred/helpers/init.py in _convert(frame)
37 for column in frame:
38 if column in c.dates:
---> 39 frame[column] = frame[column].astype('datetime64')
40 return frame
41

/usr/local/lib/python3.7/dist-packages/pandas/util/_decorators.py in wrapper(*args, **kwargs)
176 else:
177 kwargs[new_arg_name] = new_arg_value
--> 178 return func(*args, **kwargs)
179 return wrapper
180 return _deprecate_kwarg

/usr/local/lib/python3.7/dist-packages/pandas/core/generic.py in astype(self, dtype, copy, errors, **kwargs)
4999 # else, only a single dtype is given
5000 new_data = self._data.astype(dtype=dtype, copy=copy, errors=errors,
-> 5001 **kwargs)
5002 return self._constructor(new_data).finalize(self)
5003

/usr/local/lib/python3.7/dist-packages/pandas/core/internals.py in astype(self, dtype, **kwargs)
3712
3713 def astype(self, dtype, **kwargs):
-> 3714 return self.apply('astype', dtype=dtype, **kwargs)
3715
3716 def convert(self, **kwargs):

/usr/local/lib/python3.7/dist-packages/pandas/core/internals.py in apply(self, f, axes, filter, do_integrity_check, consolidate, **kwargs)
3579
3580 kwargs['mgr'] = self
-> 3581 applied = getattr(b, f)(**kwargs)
3582 result_blocks = _extend_blocks(applied, result_blocks)
3583

/usr/local/lib/python3.7/dist-packages/pandas/core/internals.py in astype(self, dtype, copy, errors, values, **kwargs)
573 def astype(self, dtype, copy=False, errors='raise', values=None, **kwargs):
574 return self._astype(dtype, copy=copy, errors=errors, values=values,
--> 575 **kwargs)
576
577 def _astype(self, dtype, copy=False, errors='raise', values=None,

/usr/local/lib/python3.7/dist-packages/pandas/core/internals.py in _astype(self, dtype, copy, errors, values, klass, mgr, **kwargs)
662
663 # _astype_nansafe works fine with 1-d only
--> 664 values = astype_nansafe(values.ravel(), dtype, copy=True)
665 values = values.reshape(self.shape)
666

/usr/local/lib/python3.7/dist-packages/pandas/core/dtypes/cast.py in astype_nansafe(arr, dtype, copy)
714 elif is_datetime64_dtype(dtype):
715 from pandas import to_datetime
--> 716 return astype_nansafe(to_datetime(arr).values, dtype, copy=copy)
717 elif is_timedelta64_dtype(dtype):
718 from pandas import to_timedelta

/usr/local/lib/python3.7/dist-packages/pandas/core/tools/datetimes.py in to_datetime(arg, errors, dayfirst, yearfirst, utc, box, format, exact, unit, infer_datetime_format, origin, cache)
465 result = _convert_and_box_cache(arg, cache_array, box, errors)
466 else:
--> 467 result = _convert_listlike(arg, box, format)
468 else:
469 result = _convert_listlike(np.array([arg]), box, format)[0]

/usr/local/lib/python3.7/dist-packages/pandas/core/tools/datetimes.py in _convert_listlike(arg, box, format, name, tz)
378 return DatetimeIndex._simple_new(values, name=name, tz=tz)
379 except (ValueError, TypeError):
--> 380 raise e
381
382 if arg is None:

/usr/local/lib/python3.7/dist-packages/pandas/core/tools/datetimes.py in _convert_listlike(arg, box, format, name, tz)
366 dayfirst=dayfirst,
367 yearfirst=yearfirst,
--> 368 require_iso8601=require_iso8601
369 )
370

pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()

pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()

pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()

pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()

pandas/_libs/tslibs/np_datetime.pyx in pandas._libs.tslibs.np_datetime.check_dts_bounds()

OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1270-01-01 00:00:00

Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True

I tied to run the following query using the fred api:
params = {'limit': 100, 'tag_names': 'daily'}
res = fr.tag.series('daily', params = params)

The objective was to get the set economic data series as a datafrane where the frequency of the data was 'daily'

But it gave an error as follows:

**ValueError Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/pandas/core/arrays/datetimes.py in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object)
1860 try:
-> 1861 values, tz_parsed = conversion.datetime_to_datetime64(data)
1862 # If tzaware, these values represent unix timestamps, so we

pandas/_libs/tslibs/conversion.pyx in pandas._libs.tslibs.conversion.datetime_to_datetime64()

ValueError: Array must be all same time zone

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last)
16 frames
/usr/local/lib/python3.6/dist-packages/pandas/core/arrays/datetimes.py in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object)
1855 dayfirst=dayfirst,
1856 yearfirst=yearfirst,
-> 1857 require_iso8601=require_iso8601
1858 )
1859 except ValueError as e:

pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()

pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()

ValueError: Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True**

I am doubting whether this is a pandas issue or something else.
I would appreciate a fix for this issue or any directions to get the afore mentioned data series of interest.

Thank you in advance,
Kind Regards,
Deshan Lokuge