Giter Site home page Giter Site logo

soccerapi's Introduction

⚠️ At the moment this project is not actively developed. Due to bookmakers' sites update some functionality may be broken. ⚠️

soccerapi

PyPI version Code style: black

Soccerapi (Application Programming Interface) is a simple wrapper build on top of some bookmakers (888sport, bet365 and Unibet) in order to get data about soccer (aka football) odds using python commands.

⚽️ The goal

The goal of the project is to provide an enjoyable way to get odds data for different soccer leagues. If you want to get these types of data you usually have to build a program by yourself (and from scratch) being able to scrape the betting site or to use some kind of paid API. Soccer API try to address this problem.

💡 The philosophy

Keep it simple. Simple API, simple http requests, few dependencies. In the past we tried to build some heavy framework, able to scrape dinamic sites (using selenium, handling complex JavaScript): was an unmaintainable nightmare.

📘 The documentation

The following section contain all the useful information to use this API at its best. Read it carefully.

Installation

Use your favorite python package manager (like pip, pipenv, poetry). For example if you use pip type in your terminal:

pip install --upgrade soccerapi

It's important to keep soccerapi updated to the last version because bookmakers sometimes change their website so soccerapi could break. We the last version on the master branch we try to keep up.


Alternatively, if you want a kind of testing/developing setup, you can install soccerapi directly from source code by first cloning the repository from GitHub and then install dev dependencies (poetry is required)

git clone https://github.com/S1M0N38/soccerapi.git
cd soccerapi
poetry install

Finally activate the environment

poetry shell

In order to obtain data from Bet365 you need to run a docker which posts on a local server a needed header to make requests to the Bet365 api. The docker run separeately from the api since it is written in JavaScript and it runs a chromedirver to run JavaScript and acquire the header.

Checkout soccerapi-server to install the docker and run it.

Usage

Import the soccerapi bookmaker, define the api variable and request odds.

from soccerapi.api import Api888Sport
# from soccerapi.api import ApiUnibet
# from soccerapi.api import ApiBet365

api = Api888Sport()
url = 'https://www.888sport.com/#/filter/football/italy/serie_a'
odds = api.odds(url)

print(odds)
[
  {
    'time': '2020-01-12T19:45:00Z'
    'home_team': 'Roma',
    'away_team': 'Juventus',
    'both_teams_to_score': {'no': 2380, 'yes': 1560},
    'double_chance': {'12': 1320, '1X': 1710, '2X': 1360},
    'full_time_resut': {'1': 3200, '2': 2160, 'X': 3450},
  },

  ...

  {
    'time': '2020-01-13T19:45:00Z'
    'home_team': 'Parma',
    'away_team': 'Lecce',
    'both_teams_to_score': {'no': 2280, 'yes': 1600},
    'double_chance': {'12': 1270, '1X': 1270, '2X': 1960},
    'full_time_resut': {'1': 1850, '2': 3850, 'X': 3800},
  }
]

The odds() method return a list of next events of the request competition (in the example: the url points to italy-serie_a)

To get a dict of valid urls that you can pass to odds() use the method competitions().

odds = api.competitions()

print(odds)
{

'Algeria': {
    'Ligue 1': 'https://www.888sport.com/#/filter/football/algeria/ligue_1',
    'Ligue 1 U21': 'https://www.888sport.com/#/filter/football/algeria/ligue_1_u21'
},

'Argentina': {
    'Primera D Metropolitana': 'https://www.888sport.com/#/filter/football/argentina/primera_d_metropolitana'
},

'Australia': {
    'A-League': 'https://www.888sport.com/#/filter/football/australia/a-league',
    'W-League (W)': 'https://www.888sport.com/#/filter/football/australia/w-league__w_'
},

...

}

This python dict is dynamically generated every time the competitions() method is run. This method crawls the bookmaker site looking for the available competitions and extract the url for every competitions offered by the bookmaker.

For some bookmakers (Bet365) many http requests are perform by competitions(), so there is the risk of receiving an IP ban. Use this method wisely (e.g. store the competitions in a json file and update them only when necessary).

The main reasons we introduced the competitions() method is due to the fact that some bookmakers (Bet365) change the url for a competitions over time in order to contrast bot scraping, so do not trust on a static list of urls for every bookmaker.

Country restriction

The regulation of online gambling varies from country to country. There are different versions of the bookmaker site depending on the provenience of your http request. Moreover, most bookmakers implement some kind of VPN detection which block VPN-http requests. Due to this constrains it's difficult to test soccerapi for worldwide usability. In the following are reported some results of the availability of bookmaker in different countries.

bet365 888sport / unibet
accessible 🇮🇹 🇧🇷 🇺🇸 🇨🇦 🇦🇺 🇧🇷 🇨🇭 🇮🇹 🇩🇪 🇩🇰 🇪🇸 🇫🇮 🇯🇵 🇳🇱 🇳🇴 🇸🇪 🇮🇪 🇮🇳 🇸🇬 🇭🇰 🇳🇿 🇲🇽 🇷🇴
inaccessible 🇫🇷 🇬🇧

Contributing

If you like to contribute to the project read CONTRIBUTING.md

soccerapi's People

Contributors

dependabot[bot] avatar muuusso avatar s1m0n38 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

soccerapi's Issues

Assertion error when odds are not available

When odds are not available for any event due to it being postponed or cancelled or the fact that the event is listed on the website but the odds and/or the category is not available, the odds are returned in the list as 'None' and since they are parsed as type:int and len ( )/3 and hence, returned as assertion error.
Can you please build an error handling/ignore into the bet365.py?
I have come across this issue many times and the only way to move on is to avoid the league altogether which is not the best of ways to go

image

Cloudfare captcha

Hello there, I have a problem with cloudfare captcha. This is table_country logged. I would be very thankfull if someone could help.
photo1644967125
.

Maximum requests limit

It seems bet365 is limiting requests.
After doing 200 requests I'm not able to establish new connection.
How to pass this?

requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb25f608a30>: Failed to establish a new connection: [Errno 111] Connection refused'))

Kambi Varnish Service Blocking IP Addresses

Hi all,

I am experiencing the following error

from soccerapi.api import Api888Sport
api = Api888Sport()
 url = 'https://www.888sport.com/#/filter/football/italy/serie_a'
odds = api.odds(url)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/soccerapi/api/base.py", line 28, in odds
    odds_to_parse = self.requests(self.url_to_competition(url))
  File "/usr/local/lib/python3.9/site-packages/soccerapi/api/888sport.py", line 40, in requests
    'full_time_result': self._request(competition, 12579),
  File "/usr/local/lib/python3.9/site-packages/soccerapi/api/888sport.py", line 65, in _request
    return self.session.get(url, params=params).json()
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 900, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I am pretty sure this is due to Kambi or Varnish blocking IP addresses after a while.

Here is an even simpler example highlighting what happens:

import requests
from typing import Dict

url: str = 'https://eu-offering.kambicdn.org/offering/v2018/888de/listView/football/germany/bundesliga.json?lang=de_DE&market=DE&client_id=2&channel_id=1&ncid=1617390885604&useCombined=true'
headers: Dict = {'Host': 'eu-offering.kambicdn.org', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0', 'Accept': 'application/json, text/javascript, */*; q=0.01', 'Accept-Encoding': 'gzip, deflate, br', 'Referer': 'https://www.888sport.com/de/fussball/', 'Origin': 'https://www.888sport.com', 'Connection': 'keep-alive'}
timeout: int = 15

response = requests.get(url, headers=headers, timeout=timeout)
print(response.text)
<!DOCTYPE html>
<html>
  <head>
    <title>410 Gone.</title>
  </head>
  <body>
    <h1>Error 410 Gone.</h1>
    <p>Gone.</p>
    <h3>Guru Meditation:</h3>
    <p>XID: 839526155</p>
    <hr>
    <p>Varnish cache server</p>
  </body>
</html>

The error occurs on a server located in DE. I can confirm that other servers (also in DE) which have not been used to fetched Kambi data do not have this issue. So this leads me to believe that it must be due to some sort of restrictions/blocking.

Anyone has any ideas on how to tackle this issue. Only thing I can think of are: Proxy, IP rotating or other such measures. Maybe someone has a better idea.

Parsing "in-play" bet365

Hi @S1M0N38 - awesome work!

How could I parse "inplay" games on bet365.com?

I saw there is a websocket where data are pushed, but there are no teams on it.

Thank you for suggestion in advance!

Odds delay

Hi,

Firstly thank you for making this, it's proving very helpful to my project!

I was wondering how large the delay is between the api and the live bet365 odds, since I am receiving different odds to those on the website.

Thanks

Bet365 X-Net-Sync-Term header

Hello @S1M0N38, some weeks ago I tested the soccerapi for Bet365 and it worked fine. Today I tried again but the requests made to get the odds had no response resulting in NoOddsError. I looked in my browser how Bet365 does that request and I noticed that in the browser they had a different header called X-Net-Sync-Term, when I added this header hard coded with the same value as my browser(ImoQYA == .8CwDX5TgJEH5wFCEu1sxh4h1RjAoyXeM6gjjlcGuetw =) the api worked again.

Do you know how this header work? I didn't found much about it. I don't know either how long this hard coded header will work.

Thanks a lot for your help, your repository is awesome!

Error when scraping Bet365

When I try to web scrape Bet365 I get the following error: JSONDecodeError: Expecting value
What I do is the following

from soccerapi.api import ApiBet365
api = ApiBet365()
competitions = api.competitions()
print(competitions)

Is anyone else able to scrape Bet365?

Data missplaced on bet365 API

Hey Simon, great work on that script!

Unibet and 888Sport works fine, but bet365 gets the data in a order that shouldn't happen.

If you get the bet365 odds it gets messed in the full_time_result getting the data for next events instead the actual draw and away odds, you can take a look in the german bundesliga 2 e.g.:

image

You can see that in the "Hamburg v Aue" event. The odds in the draw and away_team fields are actually the odds for Jahn Regensburg and Nurnberg.

Is that a known problem with bet365 data?

Thanks!

Under/Over odds in Bet365

Hello guys, first of all I wanted to thank you for your hard work.
There is an error with the under/over 2.5 odds for the Bet365 website, they are assigned the same value so I tried to correct it manually but it appears that the odds don't match the one showed in the website

Problem with competition on Bet365

Trying to use the competition method it works fine on the other two bees, but on bet365 it returns the following error:

Traceback (most recent call last):
File "c:\Users\Alessio Pepe\Desktop\api\main.py", line 4, in
odds = api.competitions()
File "C:\Users\Alessio Pepe\Desktop\api.venv\lib\site-packages\soccerapi\api\bet365.py", line 259, in competitions
table_country = self._request(
File "C:\Users\Alessio Pepe\Desktop\api.venv\lib\site-packages\soccerapi\api\bet365.py", line 308, in _request
response = requests.get('http://localhost:5000/bet365').json()
File "C:\Users\Alessio Pepe\Desktop\api.venv\lib\site-packages\requests\api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\Alessio Pepe\Desktop\api.venv\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Alessio Pepe\Desktop\api.venv\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Alessio Pepe\Desktop\api.venv\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Alessio Pepe\Desktop\api.venv\lib\site-packages\requests\adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /bet365 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000002A9C72223D0>:
Failed to establish a new connection: [WinError 10061] Impossibile stabilire la connessione. Rifiuto persistente del computer di destinazione'))

Decoding odds data

I see this library does not uses selenium or other browser automation tools to scrape data from bet365, it is interesting. Is it being maintained?

Problems scraping bet365

So I did all the ReadME instructions (installed the docker and ran it) and everything ran smoothly until I tried to call the api.competitions() method. What I got as a result of that is just: {}
I think this issue might be cloudflare related but I have no idea how to fix it. Is there anyone else facing the same problem? My code :

from soccerapi.api import ApiBet365
api = ApiBet365()
competitions = api.competitions()
print(competitions)

Bet365 is blocking Puppeteer access

I'm facing a problem with the Puppeteer library to access the Bet365 website. It just started today. Does anyone has a workaround to solve this issue?

kambi odds

most of the odds for kambi are off, only by a little bit but its a bit annoying,

Tests across the world

In this issue are collect results about the api testing across the world 🌐
Please read CONTRIBUTING.md
and produce the following form:

Country

  • bet365
  • 888sport
  • unibet

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.