jugaad-py/jugaad-data

Download live and historical data for Indian stock market

jugaad-data's Issues

Adding support for FX Derivs

Hi - is there a plan to add support for FX derivatives? If not I'll be happy to contribute. Thanks.

Is NSE Blocking IP addresses?

It appears that NSE may be blocking IP addresses. After a few hundred historical downloads, we get the below error for all dates:

Failed to download for below dates, these might be holidays, please check -
2015-01-01
2015-01-02
2015-01-05
2015-01-06
2015-01-07
2015-01-08
2015-01-09
2015-01-12
2015-01-13
2015-01-14
2015-01-15
2015-01-16
2015-01-19

Historical multiple stock data download

Can we download multiple stock historical data at once and update it daily? For example,can we download all FnO stock data (mainly OHLC) and download in Excel for last one one year or so? If yes, how??

Deliverable volume column should be added in the stock_df

Describe the required feature

Deliverable volume data is already provided on nse India 's new website and it should be added in the stock_df result also.

URL where you observed this data

https://www.nseindia.com/report-detail/eq_security

API endpoint involved

No response

NSE + BSE combined Bhavcopy data

Describe the required feature

Combined bhavcopy download for NSE and BSE for Cash segment

URL where you observed this data

https://www.bseindia.com/download/BhavCopy/Equity/EQ040322_CSV.ZIP

API endpoint involved

https://www.bseindia.com/download/BhavCopy/Equity/EQ040322_CSV.ZIP

Historical option chain from bhavcopy

Describe the required feature

Can we have option to download historical option chain when we give symbol, trading date and expiry information ? it will be really helpful

URL where you observed this data

Based on bhavcopy information

API endpoint involved

No response

Using Proxies to download Data

Hi, you might consider adding in a proxy variable when instantiating the NSELive or NSEHistory class to download data, the reason being if an individual is making a lot of requests to the NSE website for historical data or live prices - proxies come in handy so that the user does not get blocked.
I had my own version of the code to download all the data but your library is no doubt much more better.

Let me know if you are looking for more feedback!

https://2.python-requests.org/en/latest/user/advanced/#proxies

1st jan 22 to 15th feb 22 nifty index jugaad data returns unexpected result

Issue description

results returning from 2021 year. months are jumbled. dec 2021 missing.

Example Code

from datetime import date as d
from jugaad_data.nse import index_df

# Download as pandas dataframe
df = index_df(symbol="NIFTY 50", from_date=d(2022,1,1),
            to_date=d(2022,2,1))
print(df)

### Error snippet

![image](https://imgur.com/a/vkF8L7G)

Fetching data just for a few names. Error in most of the codes / names

UPDATE: It's not working for official names from NSE either

You can reproduce the code as below:

current_date = date.today()
stock_df(symbol='INFY', from_date = current_date - timedelta(days = 365), to_date = current_date, series="EQ")
        

When you get the list from this website or by another as

# ! pip install  nsetools
from nsetools import Nse

nse = Nse()
all_stocks = nse.get_stock_codes()

stocks = list(all_stocks.keys())[1:]

it won't give you for all. just a few, also sometimes it fails for HDFC too.

The exact error is

JSONDecodeError                           Traceback (most recent call last)
/tmp/ipykernel_2736/1058062146.py in <module>
      1 current_date = date.today()
----> 2 stock_df(symbol='INFY', from_date = current_date - timedelta(days = 365), to_date = current_date, series="EQ").drop(drop,axis=1)
      3 

~/anaconda3/envs/finance/lib/python3.7/site-packages/jugaad_data/nse/history.py in stock_df(symbol, from_date, to_date, series)
    168     if not pd:
    169         raise ModuleNotFoundError("Please install pandas using \n pip install pandas")
--> 170     raw = stock_raw(symbol, from_date, to_date, series)
    171     df = pd.DataFrame(raw)[stock_select_headers]
    172     df.columns = stock_final_headers

~/anaconda3/envs/finance/lib/python3.7/site-packages/jugaad_data/nse/history.py in stock_raw(self, symbol, from_date, to_date, series)
    106         chunks = ut.pool(self._stock, params)
    107 
--> 108         return list(itertools.chain.from_iterable(chunks))
    109 
    110     def derivatives_raw(self, symbol, from_date, to_date, expiry_date, instrument_type, strike_price, option_type):

~/anaconda3/envs/finance/lib/python3.7/concurrent/futures/_base.py in result_iterator()
    596                     # Careful not to keep a reference to the popped future
    597                     if timeout is None:
--> 598                         yield fs.pop().result()
    599                     else:
    600                         yield fs.pop().result(end_time - time.monotonic())

~/anaconda3/envs/finance/lib/python3.7/concurrent/futures/_base.py in result(self, timeout)
    426                 raise CancelledError()
    427             elif self._state == FINISHED:
--> 428                 return self.__get_result()
    429 
    430             self._condition.wait(timeout)

~/anaconda3/envs/finance/lib/python3.7/concurrent/futures/_base.py in __get_result(self)
    382     def __get_result(self):
    383         if self._exception:
--> 384             raise self._exception
    385         else:
    386             return self._result

~/anaconda3/envs/finance/lib/python3.7/concurrent/futures/thread.py in run(self)
     55 
     56         try:
---> 57             result = self.fn(*self.args, **self.kwargs)
     58         except BaseException as exc:
     59             self.future.set_exception(exc)

~/anaconda3/envs/finance/lib/python3.7/site-packages/jugaad_data/util.py in wrapper(*args, **kw)
    107                 if not os.path.exists(cache_dir):
    108                     os.makedirs(cache_dir)
--> 109                 j = function(**kw)
    110                 with open(path, 'wb') as fp:
    111                     pickle.dump(j, fp)

~/anaconda3/envs/finance/lib/python3.7/site-packages/jugaad_data/nse/history.py in _stock(self, symbol, from_date, to_date, series)
     73         }
     74         self.r = self._get("stock_history", params)
---> 75         j = self.r.json()
     76         return j['data']
     77 

~/anaconda3/envs/finance/lib/python3.7/site-packages/requests/models.py in json(self, **kwargs)
    908                     # used.
    909                     pass
--> 910         return complexjson.loads(self.text, **kwargs)
    911 
    912     @property

~/anaconda3/envs/finance/lib/python3.7/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    346             parse_int is None and parse_float is None and
    347             parse_constant is None and object_pairs_hook is None and not kw):
--> 348         return _default_decoder.decode(s)
    349     if cls is None:
    350         cls = JSONDecoder

~/anaconda3/envs/finance/lib/python3.7/json/decoder.py in decode(self, s, _w)
    335 
    336         """
--> 337         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338         end = _w(s, end).end()
    339         if end != len(s):

~/anaconda3/envs/finance/lib/python3.7/json/decoder.py in raw_decode(self, s, idx)
    353             obj, end = self.scan_once(s, idx)
    354         except StopIteration as err:
--> 355             raise JSONDecodeError("Expecting value", s, err.value) from None
    356         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

open interest on all strike price like nse website

Describe the required feature

hello,
code looks really good, is it possible to get full option chain with all strike price like nse so we can check where is the highest open interest on perticular stock.
it will be good to find right stocks.
thanks

URL where you observed this data

nseindia

API endpoint involved

No response

Display Buy and Sell Qty Live tick data

Request for New Feature

Hi, Can you please add 2 more elements(Buy and Sell Qty) respectively to Live tick data which may display as follows:

tick_data = n.tick_data("HDFC")
tick_data['grapthData'][0:2]

[[1611566100000, 2635,2453,1235],
 [1611566101000, 2636.25,2108,1987]]

Thanks in advance

where you observed this data

sellQty and buyQty can be viewed in premarket data, same required for live tick data.

q = n.stock_quote("HDFC")
q['preOpenMarket']

{'preopen': [{'price': 2330.55, 'buyQty': 0, 'sellQty': 57},
  {'price': 2400, 'buyQty': 0, 'sellQty': 7},
  {'price': 2408.2, 'buyQty': 0, 'sellQty': 253},
  {'price': 2449, 'buyQty': 0, 'sellQty': 10},
  {'price': 2630.45, 'buyQty': 0, 'sellQty': 0, 'iep': True},
  {'price': 2705.95, 'buyQty': 306, 'sellQty': 0},
  {'price': 2710.5, 'buyQty': 8864, 'sellQty': 0},
  {'price': 2718.9, 'buyQty': 16076, 'sellQty': 0},
  {'price': 2750, 'buyQty': 100, 'sellQty': 0}],
 'ato': {'buy': 1329, 'sell': 169},
 'IEP': 2630.45,
 'totalTradedVolume': 51122,
 'finalPrice': 0,
 'finalQuantity': 0,
 'lastUpdateTime': '25-Jan-2021 09:07:41',
 'totalBuyQuantity': 79071,
 'totalSellQuantity': 58736,
 'atoBuyQty': 1329,
 'atoSellQty': 169}

API endpoint involved

No response

Intraday Data for FNO and Stocks

Describe the required feature

NSE and BSE both do have their intraday data at EOD. Basically at EOD, we see an intraday chart we can hit the URL to get that intraday data and save it in local database/file.

URL where you observed this data

https://www.nseindia.com/get-quotes/derivatives?symbol=BANKNIFTY&identifier=OPTIDXBANKNIFTY03-02-2022PE38000.00

API endpoint involved

https://www.nseindia.com/api/chart-databyindex?index=OPTIDXBANKNIFTY03-02-2022PE38000.00

Resource not found without cookie (Major update in the website)

The history url - https://www.nseindia.com/api/historical/cm/equity?symbol=RELIANCE&series=[%22EQ%22]&from=01-01-2019&to=31-01-2019 returns "Resource not found" if proper cookies not provided in the headers

How to replicate the issue without code-
Open this url (https://www.nseindia.com/api/historical/cm/equity?symbol=RELIANCE&series=[%22EQ%22]&from=01-01-2019&to=31-01-2019) in Incognito mode, you should see Resource not found error

How to resolve-
Now in the same incognito window go to (https://www.nseindia.com/get-quotes/equity?symbol=RELIANCE) and then go back to you should see the data (https://www.nseindia.com/api/historical/cm/equity?symbol=RELIANCE&series=[%22EQ%22]&from=01-01-2019&to=31-01-2019)

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I'm getting the below error when I run your code on Heroku and I tried on desktop its working fine buy on the cloud it is displaying the error:

2020-10-01T18:20:03.659025+00:00 app[worker.1]: started.......
2020-10-01T18:20:05.585950+00:00 app[worker.1]: Traceback (most recent call last):
2020-10-01T18:20:05.586335+00:00 app[worker.1]: File "script.py", line 420, in
2020-10-01T18:20:05.586914+00:00 app[worker.1]: schedule.run_pending()
2020-10-01T18:20:05.586954+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/schedule/init.py", line 563, in run_pending
2020-10-01T18:20:05.587443+00:00 app[worker.1]: default_scheduler.run_pending()
2020-10-01T18:20:05.587478+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/schedule/init.py", line 94, in run_pending
2020-10-01T18:20:05.587736+00:00 app[worker.1]: self._run_job(job)
2020-10-01T18:20:05.591719+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/schedule/init.py", line 147, in _run_job
2020-10-01T18:20:05.592011+00:00 app[worker.1]: ret = job.run()
2020-10-01T18:20:05.592051+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/schedule/init.py", line 466, in run
2020-10-01T18:20:05.592485+00:00 app[worker.1]: ret = self.job_func()
2020-10-01T18:20:05.592525+00:00 app[worker.1]: File "script.py", line 307, in nifty_500_stocks
2020-10-01T18:20:05.592856+00:00 app[worker.1]: series="EQ")
2020-10-01T18:20:05.592892+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/jugaad_data/nse/init.py", line 170, in stock_df
2020-10-01T18:20:05.593144+00:00 app[worker.1]: raw = stock_raw(symbol, from_date, to_date, series)
2020-10-01T18:20:05.593179+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/jugaad_data/nse/init.py", line 108, in stock_raw
2020-10-01T18:20:05.593417+00:00 app[worker.1]: return list(itertools.chain.from_iterable(chunks))
2020-10-01T18:20:05.593527+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/concurrent/futures/_base.py", line 586, in result_iterator
2020-10-01T18:20:05.594038+00:00 app[worker.1]: yield fs.pop().result()
2020-10-01T18:20:05.594076+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/concurrent/futures/_base.py", line 425, in result
2020-10-01T18:20:05.594459+00:00 app[worker.1]: return self.__get_result()
2020-10-01T18:20:05.594500+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
2020-10-01T18:20:05.600236+00:00 app[worker.1]: raise self._exception
2020-10-01T18:20:05.600244+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/concurrent/futures/thread.py", line 56, in run
2020-10-01T18:20:05.600439+00:00 app[worker.1]: result = self.fn(*self.args, **self.kwargs)
2020-10-01T18:20:05.600443+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/jugaad_data/util.py", line 110, in wrapper
2020-10-01T18:20:05.600627+00:00 app[worker.1]: j = function(**kw)
2020-10-01T18:20:05.600632+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/jugaad_data/nse/init.py", line 75, in _stock
2020-10-01T18:20:05.600801+00:00 app[worker.1]: j = self.r.json()
2020-10-01T18:20:05.600805+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/requests/models.py", line 898, in json
2020-10-01T18:20:05.601212+00:00 app[worker.1]: return complexjson.loads(self.text, **kwargs)
2020-10-01T18:20:05.601237+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/json/init.py", line 354, in loads
2020-10-01T18:20:05.601530+00:00 app[worker.1]: return _default_decoder.decode(s)
2020-10-01T18:20:05.601534+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/json/decoder.py", line 339, in decode
2020-10-01T18:20:05.601754+00:00 app[worker.1]: obj, end = self.raw_decode(s, idx=_w(s, 0).end())
2020-10-01T18:20:05.601759+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/json/decoder.py", line 357, in raw_decode
2020-10-01T18:20:05.601972+00:00 app[worker.1]: raise JSONDecodeError("Expecting value", s, err.value) from None
2020-10-01T18:20:05.602034+00:00 app[worker.1]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
2020-10-01T18:20:05.951067+00:00 heroku[worker.1]: Process exited with status 1

Symbol list

If there was a function to list all possible Stocks (symbols) and Fno entities it will be good.

Timeout

Issue description

Running the bhavcopy_fo_save function always lead to timeout.

Example Code

from jugaad_data.nse import bhavcopy_save, bhavcopy_fo_save
bhavcopy_fo_save(date(2022,12,3), "D:\\Speculation\\data\\NSE_data")

Error snippet

File "D:\anaconda3\lib\site-packages\requests\models.py", line 760, in generate
    raise ConnectionError(e)

ConnectionError: HTTPSConnectionPool(host='www.nseindia.com', port=443): Read timed out.

Index Futures Live Data

Hi, is it possible to also fetch the index futures Live data, I see there is a function stock_quote_fno() which fails for BANKNIFTY and NIFTY, can a separate function be created to fetch these or add them to the existing ones?

Nifty Website not Accessible

Issue description

The website: https://niftyindices.com is not working. Is this temporary or has there been a migration to some other url. If so, the code needs to be updated to reflect that.
Specifically, I'm accessing:

jugaad-data/jugaad_data/nse/history.py

Line 275 in c6a6427

class NSEIndexHistory(NSEHistory):

.

The returned response code is 403.

Example Code

nifty_50 = index_df(symbol="NIFTY 50",
	           from_date='2023-09-01',
		   to_date='2023-09-20')

Error snippet

Traceback (most recent call last):
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/uditg/Documents/dummy/ny.py", line 391, in <module>
    main(args=parser.parse_args())
  File "/Users/uditg/Documents/dummy/ny.py", line 301, in main
    nifty_50 = index_df(symbol="NIFTY 50",
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/site-packages/jugaad_data/nse/history.py", line 371, in index_df
    raw = index_raw(symbol, from_date, to_date)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/site-packages/jugaad_data/nse/history.py", line 321, in index_raw
    return list(itertools.chain.from_iterable(chunks))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/concurrent/futures/_base.py", line 619, in result_iterator
    yield _result_or_cancel(fs.pop())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/concurrent/futures/_base.py", line 317, in _result_or_cancel
    return fut.result(timeout)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/site-packages/jugaad_data/util.py", line 109, in wrapper
    j = function(**kw)
        ^^^^^^^^^^^^^^
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/site-packages/jugaad_data/nse/history.py", line 315, in _index
    return json.loads(self.r.json()['d'])
                      ^^^^^^^^^^^^^
  File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Getting Dividend data, Corp Announcements

Describe the required feature

The Dividend data can be got from NSE but some kind of string matching will be needed, the descriptions follow majorly some kind of template however cant be sure if all the entries follow them... Another thing might be needed is that Dividends need to be split adjusted since these are just mentioned as is when its announced however it might be difficult to make use of the data if its not split adjusted... Even the bonus/split details are provided in the same corp announcements table...

URL where you observed this data

https://www.nseindia.com/companies-listing/corporate-filings-actions

API endpoint involved

No response

NSE Option chain URI not working often

Issue description

Hi @sevakram , It looks like NSE India website API has issues and has very frequent downtime. It returns empty result set in the URI in which you are hitting to fetch the option chain. In the past few weeks, it has been a frustrating issue as the automation is failing by unable to fetch expiry date. Could you fetch the expiry dates from any other reliance source API please?

Thanks for your effort.

Example Code

n = NSELive()
option_chain = n.index_option_chain("NIFTY")
expiry = option_chain['records']['expiryDates'][00]

or

option_chain = n.stock_quote_fno("NIFTY")
expiry = option_chain['expiryDates'][0]

Error snippet

Error in fetching expiry date

get list of expiry dates in a month

proposed functions-

def expiry_dates(year, month, day, instrument_type, symbol):
    """ Returns all expiry dates being actively traded on a given date for a given instrument type eg. FUTIDX, OPTIDX and symbol in a given month"""
    ----
    return list_list_of_dates

Getting Error:requests.exceptions.SSLError: HTTPSConnectionPool(host='www.nseindia.com', port=443): Max retries exceeded with url

It worked fine yesterday but today it is showing error. This is the only python library that working fine with historical data without much code. Please sort out this issue thanks.

My code failed (Sample bug)

Issue description

Some issue happened

Example Code

import os
os.listdir()

Error snippet

Exception: Could not find directory

jugaad-data 0.24 requires click==7.1.2, but you have click 8.1.3 which is incompatible.

Issue description

I am getting the following error when installing the package. Should that be >7.1.2 ? Or what do I do about it?
I am running python 3.7.12 on win7.
jugaad-data 0.24 requires click==7.1.2, but you have click 8.1.3 which is incompatible.

Example Code

pip install spyder==4.2.0

Error snippet

jugaad-data 0.24 requires click==7.1.2, but you have click 8.1.3 which is incompatible.

Option chain data downloading issue

Issue description

Can't download option chain data. Screenshot attached.

Example Code

from jugaad_data.nse import derivatives_csv, derivatives_df
df = derivatives_df(symbol="SBIN", from_date=date(2022,1,1), to_date=date(2022,2,24),
            expiry_date=date(2022,3,31), instrument_type="OPTSTK", option_type="CE", strike_price=500)
print(df.head())

Error snippet

JSONDecodeError                           Traceback (most recent call last)
[<ipython-input-42-2a115219c3d7>](https://eei938yvh5k-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20220328-060046-RC02_437851183#) in <module>()
      1 from jugaad_data.nse import derivatives_csv, derivatives_df
      2 df = derivatives_df(symbol="SBIN", from_date=date(2022,1,1), to_date=date(2022,2,24),
----> 3             expiry_date=date(2022,3,31), instrument_type="OPTSTK", option_type="CE", strike_price=500)
      4 print(df.head())

11 frames
[/usr/lib/python3.7/json/decoder.py](https://eei938yvh5k-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20220328-060046-RC02_437851183#) in raw_decode(self, s, idx)
    353             obj, end = self.scan_once(s, idx)
    354         except StopIteration as err:
--> 355             raise JSONDecodeError("Expecting value", s, err.value) from None
    356         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Error in kaggle notebook

Getting error in Kaggle notebook

/opt/conda/lib/python3.7/site-packages/jugaad_data/util.py in wrapper(*args, **kw)
    102             if not os.path.isfile(path):
    103                 if not os.path.exists(cache_dir):
--> 104                     os.makedirs(cache_dir)
    105                 j = function(**kw)
    106                 with open(path, 'wb') as fp:

/opt/conda/lib/python3.7/os.py in makedirs(name, mode, exist_ok)
    219             return
    220     try:
--> 221         mkdir(name, mode)
    222     except OSError:
    223         # Cannot rely on checking for EEXIST, since the operating system

FileExistsError: [Errno 17] File exists: '/root/.cache/nsehistory-stock'

Holidays list is missing a date

For some reason the holidays list doesn't include 25/05/2020, I appended it manually for my use case.

You could add it to your library and complete it 👍

Vix returns nothing on n.live_index('INDIA VIX')

print(n.live_index('INDIA VIX')) returns {} even though 'INDIA VIX' shows up when using print(n.all_indices())
also how do i get data regarding usdinr fut ?

Wonderful job! Is there any way to get the ltp data for a single option ? i.e Banknifty weekly option LTP tick by tick

Live data and collection of all top 50 companies

Issue description

from jugaad_data.nse import NSELive
n = NSELive()
q = n.stock_quote("HDFC")
print(q['priceInfo'])
How can i get live data only and also collection of all top 50 companies means names of companies

Example Code

from jugaad_data.nse import NSELive
n = NSELive()
q = n.stock_quote("HDFC")
print(q['priceInfo'])

Error snippet

from jugaad_data.nse import NSELive
n = NSELive()
q = n.stock_quote("HDFC")
print(q['priceInfo'])

'index_option_chain' function returns empty data set

NSELive().index_option_chain("NIFTY") returns empty result set {}. Suddenly it stopped working and returns empty result set and doesn't return expiry date.

Daily Downloading Participant wise Open Interest file

Describe the required feature

Daily Download of Full bhavcopy and FO File is working great.
Can you please add feature for downloading participant wise open interest file?

https://archives.nseindia.com/content/nsccl/fao_participant_oi_04052022.csv

URL where you observed this data

https://archives.nseindia.com/content/nsccl/fao_participant_oi_04052022.csv

API endpoint involved

No response

Base URL needs to be updated

Base URL in NSEArchies class:

base_url = "https://www1.nseindia.com"

Jugaad Data - missing data

When I tried downloading NIFTY 50 index data of 2019, only 49 days data was available. Similar problem is there in 2017, 2018, 2019, 2020 and 2021.

Can we get Tik by Tik Data?

Hi

Can you please tell me if we can get the tik by tik data.

Missing data while pulling historical stock prices

Issue description

Below is the code for downloading historical stock data for a company to a CSV file

from datetime import date from jugaad_data.nse import stock_csv stock_csv(symbol="NATIONALUM", from_date=date(2020,1,1), to_date=date(2023,1,18), series="EQ", _output="D:\\Stock\\NALUM.csv")

NSE Symbol : NATIONALUM
From Date : 1/1/2020
To Date : 1/18/2023

This should've pulled the data for all the 760 trading days in that time range into the CSV file. However the data is missing for the time period 10/6/2021 to 31/5/2022

This isn't a standalone issue for this particular company. All of them have the same issue of missing data for a particular period within the expected time range
nationalum.txt
NALUM.csv

Example Code

from datetime import date
from jugaad_data.nse import stock_csv
            
# Download data and save to a csv file
stock_csv(symbol="NATIONALUM", from_date=date(2020,1,1),
            to_date=date(2023,1,18), series="EQ", output="D:\\Stock\\NALUM.csv")

Error snippet

Not an error. Missing data in the output file as explained in the description

zipfile.BadZipFile: File is not a zip file for bhavcopy_fo_save

Traceback (most recent call last):
  File "/home/mb/my/career/market/bhav/setup.py", line 8, in <module>
    bhavcopy_fo_save(date(2021,7,20), ".")
  File "/home/mb/my/career/market/bhav/venv/lib/python3.9/site-packages/jugaad_data/nse/archives.py", line 120, in bhavcopy_fo_save
    text = self.bhavcopy_fo_raw(dt)
  File "/home/mb/my/career/market/bhav/venv/lib/python3.9/site-packages/jugaad_data/nse/archives.py", line 13, in unzipper
    with zipfile.ZipFile(file=fp) as zf:
  File "/usr/lib/python3.9/zipfile.py", line 1257, in __init__
    self._RealGetContents()
  File "/usr/lib/python3.9/zipfile.py", line 1324, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Issue description

The error comes up only when one makes two or more calls to the fetch_stock_data() function. This has been coming at a higher frequency for longer periods of time. Tried it multiple times after creating a thread like in the Issue in October 2020 but not resolving.

Example Code

from jugaad_data.nse import stock_csv, stock_df

def fetch_stock_data(symbol, start_year, start_month, start_day, end_year, end_month, end_day, series):
    """
        Function to fetch historical stock data of specific symbols for a given range of dates
        Input: Stock Symbol and the date range
        Output: Stock Historical Data in form of a data frame
    """
    df = stock_df(symbol=str(symbol), 
    from_date=date(start_year,start_day,start_month),to_date=date(end_year,end_month,end_day), 
    series=str(series))
    return df

techm_df = fetch_stock_data("TECHM", 2010, 1, 1, 2021, 12, 31, "EQ")

Error snippet

\local\programs\python\python39\lib\json\__init__.py in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    344             parse_int is None and parse_float is None and
    345             parse_constant is None and object_pairs_hook is None and not kw):
--> 346         return _default_decoder.decode(s)
    347     if cls is None:
    348         cls = JSONDecoder

Error in fetching expiry dates

Hi, The library was working properly till day before yesterday and fetching the expiry dates as mentioned in the documentation. However, now it is through the error message and not fetching the expiry dates. Following is the code that I was using to fetch the current expiry date. It looks like in the results returned, there is no any key with the name 'records'. Only 'filtered' key with some values is being returned. Is it any API change in the NSE source or problem with Jugaad library? Please help.

n = NSELive()
option_chain = n.index_option_chain("BANKNIFTY")
expiry = option_chain['records']['expiryDates'][00]

Refreshing and getting live data

How often can the live data be fetched (like minute intervals). Also, could you please elaborate how to fetch that at a minute basis through code for say options.

error (demo_crushed) not found

VOLUME & TIME PERIOD

Is it possible to add volume as it is very important and time period like 15 minutes, 1 hour, 1 day etc,

Not able to download historical NIFTY50 index data

Issue description

I am trying to download the historical NIFTY50 index data. I am getting an error given below.
However, the individual stocks data download for the same period are working fine.

Please find the example code as well as the error snippet below.

Example Code

from jugaad_data.nse import index_csv
# Download as csv
index_csv(symbol="NIFTY 50", from_date=date(2020,1,1),
            to_date=date(2020,1,30), output=r"C:\Users\Kart\Documents\Workspace\pypen\index_data.csv")

Error snippet

Traceback (most recent call last):
  File "C:\Users\Kart\Documents\Workspace\pypen\lib\site-packages\requests\models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Kart\Documents\Workspace\pypen\finance\bhavcopy.py", line 26, in <module>
    index_csv(symbol="NIFTY 50", from_date=date(2020,1,1),
  File "C:\Users\Kart\Documents\Workspace\pypen\lib\site-packages\jugaad_data\nse\history.py", line 336, in index_csv
    raw = index_raw(symbol, from_date, to_date)
  File "C:\Users\Kart\Documents\Workspace\pypen\lib\site-packages\jugaad_data\nse\history.py", line 318, in index_raw
    return list(itertools.chain.from_iterable(chunks))
  File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\_base.py", line 600, in result_iterator
    yield fs.pop().result()
  File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\_base.py", line 433, in result
    return self.__get_result()
  File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\_base.py", line 389, in __get_result
    raise self._exception
  File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "C:\Users\Kart\Documents\Workspace\pypen\lib\site-packages\jugaad_data\util.py", line 109, in wrapper
    j = function(**kw)
  File "C:\Users\Kart\Documents\Workspace\pypen\lib\site-packages\jugaad_data\nse\history.py", line 312, in _index
    return json.loads(self.r.json()['d'])
  File "C:\Users\Kart\Documents\Workspace\pypen\lib\site-packages\requests\models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

URL Changed By NSE to nsearchives.nseindia.com Please Update this ASAP.

https://github.com/jugaad-py/jugaad-data/blame/c6a6427dc8041ef26c086690c85c46f2d83dcf2a/jugaad_data/nse/archives.py#L25C5-L25C47

New Base URL to be updates is : https://nsearchives.nseindia.com/

jugaad-data/jugaad_data/nse/archives.py

Line 24 in c6a6427

class NSEArchives:

class NSEArchives:
base_url = "https://archives.nseindia.com"
"""Conventions

Is this Adjusted close data?

Hi the historical bhav copy files, will not take care of the any adjustments. Does the module take care of splits, etc?

sample issue

Example Code

from datetime import date
from jugaad_data.nse import bhavcopy_save, bhavcopy_fo_save

# Download bhavcopy
bhavcopy_save(date(2020,1,1), "/path/to/directory")

# Download bhavcopy for futures and options
bhavcopy_fo_save(date(2020,1,1), "/path/to/directory")

# Download stock data to pandas dataframe
from jugaad_data.nse import stock_df
df = stock_df(symbol="SBIN", from_date=date(2020,1,1),
            to_date=date(2020,1,30), series="EQ")

Different Timeframes?

Describe the required feature

I want to get data for different timeframes like 5m, 10m, 1h, so on.

URL where you observed this data

https://www.tradingview.com/chart/?symbol=NSE%3ANIFTY

API endpoint involved

No response

Http2 Support (HTTPX)

Describe the required feature

jugad-data is using requests library does not support http2 but now nse support http2 protocol which faster and more efficient.
Kindly use httpx library instead of requests.

URL where you observed this data

https://www.python-httpx.org/http2/

API endpoint involved

No response

Historical data is not consistent for few tickers.

Hello Team,

This below code fetches data for ABBOTINDIA till 29th June 2021
df = stock_df(symbol = 'ABBOTINDIA', from_date=date(2021, 5, 1), to_date=date.today(),series="EQ")
But the same code fetches data till 28th June for 3MINDIA
df = stock_df(symbol = 'ABBOTINDIA', from_date=date(2021, 5, 1), to_date=date.today(),series="EQ")

While trying to extract data for ADANIGREEN, it gives OHLC data till June 08th, 2021

Full Bhavcopy Data for 2019 throwing exception

Hi, I think better exception handling can be done in your code right now. For some reason, I cannot fetch any full_bhavcopy_save for 2019, and for prior years, I believe the data is not present anymore on https://archives.nseindia.com/products/content/sec_bhavdata_full_{ddmmyyyy}.csv

Are you planning to put any fix in there? Details of Exception below.

Traceback (most recent call last):
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\urllib3\response.py", line 438, in _error_catcher
yield
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\urllib3\response.py", line 519, in read
data = self._fp.read(amt) if not fp_closed else b""
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\http\client.py", line 458, in read
n = self.readinto(b)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\http\client.py", line 502, in readinto
n = self.fp.readinto(b)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\socket.py", line 669, in readinto
return self._sock.recv_into(b)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\models.py", line 753, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\urllib3\response.py", line 576, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\urllib3\response.py", line 541, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\contextlib.py", line 131, in exit
self.gen.throw(type, value, traceback)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\urllib3\response.py", line 443, in _error_catcher
raise ReadTimeoutError(self._pool, None, "Read timed out.")
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='www.nseindia.com', port=443): Read timed out.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "bhavcopy_downloader.py", line 29, in
full_bhavcopy_save(dates, savepath)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\jugaad_data\nse\archives.py", line 91, in full_bhavcopy_save
text = self.full_bhavcopy_raw(dt)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\jugaad_data\nse\archives.py", line 81, in full_bhavcopy_raw
r = self.get("bhavcopy_full", yyyy=yyyy, mm=mm, dd=dd)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\jugaad_data\nse\archives.py", line 52, in get
self.r = self.s.get(url, timeout=self.timeout)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\sessions.py", line 677, in send
history = [resp for resp in gen]
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\sessions.py", line 677, in
history = [resp for resp in gen]
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\sessions.py", line 237, in resolve_redirects
resp = self.send(
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\sessions.py", line 697, in send
r.content
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\models.py", line 831, in content
self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\models.py", line 760, in generate
raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.nseindia.com', port=443): Read timed out.