gerardcl/renfe-cli

Python CLI written in Rust for fast Renfe search website trains timetables retrieval

Renfe Timetables CLI

Get faster Renfe trains timetables in your terminal, with Python3.7+ support. No longer need to open the browser! Just keep using your terminal 😀

renfe-cli is written in Rust (since v4.0.0) and published to pypi.org as a Python package (CLI and library).

See the changelog.

NOTE since I am more often using Rodalies trains I have created rodalies-cli. I hope you like it too!

Installation

Install Python CLI package renfe-cli

pip install renfe-cli --upgrade

Usage (CLI)

This CLI behaves as a person/bot going through the official renfe.com search site, using headless chrome browser. If the headless chrome browser is not found it will be downloaded.

The navigation through the site happens in the following steps:

1. Writes down and selects origin station
2. Writes down and selects destination station
3. Writes down and selects the day to search for
4. Writes down and selects the month to search for
5. Writes down and selects the year to search for
6. Clicks on search button
7. Parses the HTML data, optionally sorts the connections and prints the timetable

$ renfe-cli -h
Usage: renfe-cli [options]

Options:
    -f ORIGIN           Set From origin station
    -t DESTINATION      Set To destination station
    -d, --day DAY       Set Day to search timetable for (default: today)
    -m, --month MONTH   Set Month to search timetable for (default: today's month)
    -y, --year YEAR     Set Year to search timetable for (default: today's year)
    -w, --wait SECONDS  Set Wait time in seconds for Renfe search result page (default: 2)
    -s, --sort          Option to sort the timetable by Duration
    -h, --help          Print this help menu

Getting the timetable

Let's show an example of minimal inputs (origin and destination stations) with specific date:

$ renfe-cli -f Barc -t Mad -d 27
Loading stations from Renfe web
Provided input 'Barc' station matches with 'Barcelona (ALL) '...continue
Provided input 'Mad' station matches with 'Madrid (ALL) '...continue
Today is: 2023-11-26
Searching timetable for date: 2023-11-27
loading headless chrome browser
navigating to renfe timetable search page
waiting for search page
adding origin station
adding destination station
adding day
adding month
adding year
searching timetable
got timetable page
loading timetable
=========================TIMETABLE=========================
Train        |   Departure  |   Arrival    | Duration
-----------------------------------------------------------
AVE          |    05.50     |    09.10     | 3 h. 20 min.
-----------------------------------------------------------
AVE          |    06.20     |    08.50     | 2 h. 30 min.
-----------------------------------------------------------
AVLO         |    06.35     |    09.20     | 2 h. 45 min.
-----------------------------------------------------------
AVE          |    07.00     |    09.30     | 2 h. 30 min.
-----------------------------------------------------------
AVE          |    07.40     |    10.10     | 2 h. 30 min.
-----------------------------------------------------------
LD-AVE       |    07.45     |    15.35     | 7 h. 50 min.
-----------------------------------------------------------
AVE          |    08.00     |    11.12     | 3 h. 12 min.
-----------------------------------------------------------
AVE          |    08.25     |    10.55     | 2 h. 30 min.
-----------------------------------------------------------
REG.EXP.     |    08.43     |    18.09     | 9 h. 26 min.
-----------------------------------------------------------
AVE          |    09.00     |    11.45     | 2 h. 45 min.
-----------------------------------------------------------
AVLO         |    10.00     |    13.17     | 3 h. 17 min.
-----------------------------------------------------------
AVE          |    11.00     |    13.45     | 2 h. 45 min.
-----------------------------------------------------------
AVE          |    12.00     |    15.12     | 3 h. 12 min.
-----------------------------------------------------------
AVE INT      |    12.50     |    15.45     | 2 h. 55 min.
-----------------------------------------------------------
AVE          |    13.25     |    15.54     | 2 h. 29 min.
-----------------------------------------------------------
AVE          |    14.00     |    17.12     | 3 h. 12 min.
-----------------------------------------------------------
AVLO         |    15.00     |    17.45     | 2 h. 45 min.
-----------------------------------------------------------
AVE          |    15.25     |    17.55     | 2 h. 30 min.
-----------------------------------------------------------
AVE          |    16.00     |    19.12     | 3 h. 12 min.
-----------------------------------------------------------
AVE          |    16.25     |    18.55     | 2 h. 30 min.
-----------------------------------------------------------
AVE          |    17.00     |    19.45     | 2 h. 45 min.
-----------------------------------------------------------
AVE          |    17.25     |    19.55     | 2 h. 30 min.
-----------------------------------------------------------
AVE          |    18.00     |    21.12     | 3 h. 12 min.
-----------------------------------------------------------
AVE          |    18.25     |    20.55     | 2 h. 30 min.
-----------------------------------------------------------
AVE          |    18.40     |    21.45     | 3 h. 5 min.
-----------------------------------------------------------
AVE          |    19.25     |    21.55     | 2 h. 30 min.
-----------------------------------------------------------
AVE          |    20.00     |    23.12     | 3 h. 12 min.
-----------------------------------------------------------
AVLO         |    21.00     |    23.45     | 2 h. 45 min.
-----------------------------------------------------------
AVE          |    21.25     |    23.55     | 2 h. 30 min.
===========================================================

Usage (Library)

renfe-cli can be imported as a python package into your project, offering utilities when willing to deal with the Renfe search web site.

$ python
Python 3.8.18 (default, Aug 25 2023, 13:20:30)
[GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import renfe_cli
>>> renfe = renfe_cli.
renfe_cli.Renfe(             renfe_cli.main(              renfe_cli.print_timetable(   renfe_cli.renfe_cli          renfe_cli.search_timetable(
>>> renfe = renfe_cli.Renfe()
Loading stations from Renfe web
>>> renfe.
renfe.filter_station(  renfe.stations_match(
>>> renfe.stations_match("Bar")
['Barcelona (ALL) ', 'Padrón-Barbanza']

---

Contribute or Report with Issues

If Renfe's website is changed or you find any issue to be fixed or nice enhancements to have, please: create an issue.

Development

This project makes use of Rust bindings for the Python interpreter thanks to pyo3. It is already available as a dependency.

To develop, build and publish, this project makes use of maturin project. See usage.

Example of first time working with this repository:

$ git clone https://github.com/gerardcl/renfe-cli.git && cd renfe-cli
$ python -m venv venv
$ . venv/bin/activate
$ pip install -U pip
$ pip install -U maturin
$ maturin develop
🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
🐍 Not using a specific python interpreter
📡 Using build options features from pyproject.toml
   Compiling renfe-cli v4.1.0 (/path/to/renfe-cli)
    Finished dev [unoptimized + debuginfo] target(s) in 7.07s
📦 Built wheel for abi3 Python ≥ 3.7 to /tmp/.tmpDsjowL/renfe_cli-4.1.0-cp37-abi3-linux_x86_64.whl
🛠 Installed renfe-cli-4.1.0

Maturin takes care of compiling the rust code, generating the bindings for python and installing the package for local use (as library or binary/CLI).

renfe-cli's Issues

Enable version flag and version status check

Is your feature request related to a problem? Please describe.
It would be nice to show the current version when running renfe-cli --version/-v and check if running on latest version.

Describe the solution you'd like
Enable the version flag and prompt the check of version.

Describe alternatives you've considered
None

Additional context
Inspiration from rodalies-cli

Travis CI deploy to PyPI

Current .travis.yml works for Linux without RST convertion.
To solve:

• Missing right Windows and OSX deploy stage.
• Missing package pypandoc in all three target OS.

See issue here: https://travis-ci.org/gerardcl/renfe-cli/builds/528291717

maintenance April 2024

Is your feature request related to a problem? Please describe.
General dependencies maintenance.

Describe the solution you'd like
cargo update

Describe alternatives you've considered
N/A

Additional context
Related to the rustls findings.

defaults overwritting when selecting different origin or destination

macOS build fails

Describe the bug
TravisCI fails on macOS pipeline

Expected behavior
Should not fail

Screenshots

$ pip3 install requests nose --upgrade

Collecting requests

  Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB)

     |████████████████████████████████| 61kB 6.4MB/s 

Collecting nose

  Downloading https://files.pythonhosted.org/packages/15/d8/dd071918c040f50fa1cf80da16423af51ff8ce4a0f2399b7bf8de45ac3d9/nose-1.3.7-py3-none-any.whl (154kB)

     |████████████████████████████████| 163kB 9.7MB/s 

Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1

  Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB)

     |████████████████████████████████| 133kB 12.1MB/s 

Collecting certifi>=2017.4.17

  Downloading https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl (154kB)

     |████████████████████████████████| 163kB 14.9MB/s 

Collecting idna<2.9,>=2.5

  Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)

     |████████████████████████████████| 61kB 11.6MB/s 

Collecting chardet<3.1.0,>=3.0.2

  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)

     |████████████████████████████████| 143kB 19.6MB/s 

Installing collected packages: urllib3, certifi, idna, chardet, requests, nose

ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/man'

Consider using the `--user` option or check the permissions.

The command "pip3 install requests nose --upgrade" failed and exited with 1 during .

Your build has been stopped.

Environment (please complete the following information):

• OS: macOS
• Version: 2.0.0

Additional context
Solution might be to install dependency packages at user level

CLI exits if Firefox is not installed

Describe the bug
The CLI exits execution if Firefox is not installed in the system (Mac in my case)

To Reproduce
Steps to reproduce the behavior:

1. Install CLI on Mac
2. Check firefox is not installed
3. run cli:

$ renfe-cli -t 92102

Expected behavior
trains are retrieved

Current behaviour: CLI exits with Python stacktrace:

Today is: 2021-10-19
Searching timetable for date: 2021-10-19
From SILS to TOLEDO
Be patient, navigating through renfe site now...
/bin/sh: /Applications/Firefox.app/Contents/MacOS/firefox: No such file or directory
Traceback (most recent call last):
  File "/usr/local/bin/renfe-cli", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/renfe/cli.py", line 33, in main
    times = get_timetable(origin_name, destination_name, int(options.days), options.browser, int(options.search_timeout))
  File "/usr/local/lib/python3.9/site-packages/renfe/timetable.py", line 14, in get_timetable
    soup = get_soup(browser, origin, destination, days_from_today, search_timeout)
  File "/usr/local/lib/python3.9/site-packages/renfe/timetable.py", line 42, in get_soup
    browser = get_browser(browser)
  File "/usr/local/lib/python3.9/site-packages/renfe/timetable.py", line 29, in get_browser
    browser = webdriver.Firefox(executable_path=GeckoDriverManager().install(), firefox_options=firefox_options)
  File "/usr/local/lib/python3.9/site-packages/webdriver_manager/firefox.py", line 22, in __init__
    self.driver = GeckoDriver(version=version,
  File "/usr/local/lib/python3.9/site-packages/webdriver_manager/driver.py", line 83, in __init__
    self.browser_version = firefox_version()
  File "/usr/local/lib/python3.9/site-packages/webdriver_manager/utils.py", line 174, in firefox_version
    raise ValueError(f'Could not get version for Firefox with this command: {cmd}')
ValueError: Could not get version for Firefox with this command: /Applications/Firefox.app/Contents/MacOS/firefox --version

Screenshots

Environment (please complete the following information):

• OS: Mac OS
• Version:

Additional context
I tried passing -b chrome to avoid using default browser, but then it fails with another error (a CSS ID that does not exist anymore)

travis fails on Windows job when choco installs python

Describe the bug
renfe-cli is python 3.7.x ready --> windows travis job installs a new choco version released of python (3.8)

To Reproduce
Steps to reproduce the behavior:
when a new build of renfe-cli on travis, windows job fails due to a new python release provided by default via choco install

Expected behavior
choco keeps installing python 3.7

Environment (please complete the following information):

• OS: Windows
• Version: 2.0.0

Additional context
Update travis file to choco installing python 3.7

add package documentation and automate creation in setup.py

Renfe Data

Describe the solution you'd like
Maybe you know it and you have choosen to scrape renfe site instead but there is a open json data of Renfe Schedules and stations in this link:

• https://data.renfe.com/dataset?tags=horarios

Describe alternatives you've considered
I have considered use it but due I am living in North and Feve does not respect schedules at all it is not an option for me. But, there is a platform called "Elcano" which can provide realtime data of the trains.

• https://www.adif.es/w/elcano

I have been sniffing traffic because there is no info to use it by public people but you can scrape adif, it provides realtime just for current train which makes sense but if the train is stopped at any station waiting for other (because there is only one rail) it doesn't provide that info just update the train arriving time after some time.

The api in phones use firebase to register an id for the user (I think based on installation id) and later you can access elcano api but I am not really sure about all this process because I haven't tested yet. The difference is that in phone you can view almost real position of the train and not only the time to arrive in the station.

Additional context

Hope this info is helpful.

Enable verbosity levels

Is your feature request related to a problem? Please describe.
Would be good to let user define some verbosity levels

Describe the solution you'd like
I see three levels:

• INFO: minimal required info
• DEBUG: with more verbosity on steps
• VISUAL: prompting screenshots on the terminal on the screens one is reaching on each step (useful for reporting bugs)

Describe alternatives you've considered
None

Additional context
None

Enable origin and destination checks

Is your feature request related to a problem? Please describe.
It would be nice to have a check on the values provided as origin and destination stations from existing list of stations from the Renfe search web site.

Describe the solution you'd like
Enable check on origin and destination station, and provide error when not found or matching many stations.

Describe alternatives you've considered
None

Additional context
None

Compatibilitat amb linux

Seria genial poder gaudir d'aquesta aplicació a també linux.

Soporte versiones Chrome superiores a 114

Describe the bug
Debido a un cambio donde se almacenan los drivers de Chrome, la librería solo es compatible con Chrome 114 o inferior.

To Reproduce
Instalar la versión estable de Chrome a 2023-09-04 (116.0.5845) y ejecutar la aplicación. El error es:

ERROR:root:There is no such driver by url https://chromedriver.storage.googleapis.com/LATEST_RELEASE_116.0.5845

Environment (please complete the following information):

• OS: Ubuntu
• Version: 22.04

Additional context
Parece que los desarrolladores de Chrome cambiaron la localización de los drivers. Más contexto aquí: https://chromedriver.chromium.org/getting-started
Actualizar Selenium y retocar ligeramente el código parece una posible solución:
https://stackoverflow.com/questions/76724939/there-is-no-such-driver-by-url-https-chromedriver-storage-googleapis-com-lates

En mi caso, he probado a descargar el driver manualmente de https://googlechromelabs.github.io/chrome-for-testing/ y añadirlo al PATH, sin éxito. Si ha funcionado hacer downgrade de la versión de Chrome a la 114:

• https://makandracards.com/makandra/486433-how-to-downgrade-google-chrome-in-ubuntu
• https://www.ubuntuupdates.org/package/google_chrome/stable/main/base/google-chrome-stable?id=202706&page=2

Enable wait time for search results page

Is your feature request related to a problem? Please describe.
Before v4.0.0 renfe-cli had a waiting time when navigating to search result page. This helped ensuring we loaded the full page before parsing it.

Describe the solution you'd like
Set it configurable.

Describe alternatives you've considered
None

Additional context
Default to 2 seconds.

Defaults for date params

Is your feature request related to a problem? Please describe.
Before v4.0.0 renfe-cli had defaults for date values (i.e.: day, month and year).

Describe the solution you'd like
To have this feature back.

Describe alternatives you've considered
None

Additional context
Default to today.

Navigation on macOS requires use of current elem

Describe the bug
When running the CLI on macOS the navigation gets stopped as it is using tab actions.

To Reproduce
Steps to reproduce the behavior:

1. run the latest CLI version
2. fails after first input

Expected behavior
Use of actions over current elements instead of general tab actions.

Screenshots
N/A

Environment (please complete the following information):

• OS: macOS
• Version: sonoma

Additional context
This might be affecting other OSes

Headless Chrome can't find button to trigger the search

Describe the bug
When running the CLI one runs into an error indicating that Headless Chrome wasn't able to find the button to trigger the search:

searching timetable
thread '<unnamed>' panicked at src/timetable.rs:137:10:
called `Result::unwrap()` on an `Err` value: Method call error -32000: Invalid search result range
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
  File "foo/renfe-cli", line 8, in <module>
    sys.exit(main())
             ^^^^^^
pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: Method call error -32000: Invalid search result range

They might have changed their HTML and the selector that is used isn't valid anymore.

To Reproduce
Run renfe-cli -f Barc -t Mad

Expected behavior
The renfe-cli doesn't panic and prints out a timetable.

Environment (please complete the following information):

• OS: macOS M1
• Version: 4.2.0

Additional context
Add any other context about the problem here.

Refactor keeping functionalities but using Rust as backend

Is your feature request related to a problem? Please describe.
We keep seeing a lot of inconsistencies and hard management when being under the python context of this project.
We have gained a lot of motivation with rodalies-cli.

Describe the solution you'd like
Refactor the whole python project to use Rust as backend but keep being a python package (library and script).

Describe alternatives you've considered
Fight with all inconsistencies within the dependencies required on the current python project. No go.

Additional context
We will use pyo3 for Rust into python bindings and manage the project with maturin.

Option to sort trains by duration

Is your feature request related to a problem? Please describe.
Users might want to have a sorted table that allows the to see connection with the shortest duration on top.

Describe the solution you'd like
Add a flag -s/--sorted that will sort the timetable by duration.

enable codec MD to RST for pypi page

change origin and destination defaults through config

Maintenance upgrade with latest pyo3 release

Is your feature request related to a problem? Please describe.
This is a general maintenance on dependencies with improvement on performance.

Describe the solution you'd like
Maintenance on dependencies and fix on related deprecation notices.

Describe alternatives you've considered
N/A

Additional context
General maintenance.

Posible cambio en la web de Renfe

Describe the bug
Parece que Renfe ha retocado ligeramente la web y al intentar obtener los horarios se obtiene el siguiente error:

Traceback (most recent call last):
  File "/home/alberto/miniconda3/envs/renfe-cli/bin/renfe-cli", line 8, in <module>
    sys.exit(main())
  File "/home/alberto/miniconda3/envs/renfe-cli/lib/python3.9/site-packages/renfe/cli.py", line 33, in main
    times = get_timetable(
  File "/home/alberto/miniconda3/envs/renfe-cli/lib/python3.9/site-packages/renfe/timetable.py", line 19, in get_timetable
    soup = get_soup(browser, origin, destination, days_from_today, search_timeout)
  File "/home/alberto/miniconda3/envs/renfe-cli/lib/python3.9/site-packages/renfe/timetable.py", line 64, in get_soup
    origin_option = browser.find_element_by_css_selector("#awesomplete_list_1_item_0")
  File "/home/alberto/miniconda3/envs/renfe-cli/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 808, in find_element_by_css_selector
    return self.find_element(by=By.CSS_SELECTOR, value=css_selector)
  File "/home/alberto/miniconda3/envs/renfe-cli/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 1244, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "/home/alberto/miniconda3/envs/renfe-cli/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 424, in execute
    self.error_handler.check_response(response)
  File "/home/alberto/miniconda3/envs/renfe-cli/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"#awesomplete_list_1_item_0"}
  (Session info: headless chrome=114.0.5735.198)
Stacktrace:
#0 0x5618ce2c64e3 <unknown>
#1 0x5618cdff5c76 <unknown>
#2 0x5618ce031c96 <unknown>
#3 0x5618ce031dc1 <unknown>
#4 0x5618ce06b7f4 <unknown>
#5 0x5618ce05103d <unknown>
#6 0x5618ce06930e <unknown>
#7 0x5618ce050de3 <unknown>
#8 0x5618ce0262dd <unknown>
#9 0x5618ce02734e <unknown>
#10 0x5618ce2863e4 <unknown>
#11 0x5618ce28a3d7 <unknown>
#12 0x5618ce294b20 <unknown>
#13 0x5618ce28b023 <unknown>
#14 0x5618ce2591aa <unknown>
#15 0x5618ce2af6b8 <unknown>
#16 0x5618ce2af847 <unknown>
#17 0x5618ce2bf243 <unknown>
#18 0x7f38bb974b43 <unknown>

To Reproduce
Ejecutar renfe-cli -o 17000 -t 31400 -b "chrome" con Chrome 114 instalado

Expected behavior
Que se muestren los horarios

Environment (please complete the following information):

• OS: Ubuntu
• Version: 22.04