Unable to retrieve any list

[gilesdoy]

Unable to retrieve any list

[JJHeathview]

Hi Phil, I get the same error message, the error started on the 2-Dec, do you think it is an issue with my version of RCurl? Regards, Jason

…

________________________________ From: gilesdoy <[email protected]> Sent: 07 December 2020 20:01 To: phillc73/abettor <[email protected]> Cc: Subscribed <[email protected]> Subject: [phillc73/abettor] Unable to retrieve any list (#25) Hi, I've received this error when running any of the 'list' commands. Any ideas what the bug might be? listCompetitions(eventTypeIds=1, toDate = (format(Sys.time() + 86400 * 365, "%Y-%m-%dT%TZ")) ) Error in function (type, msg, asError = TRUE) : error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://eur05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fphillc73%2Fabettor%2Fissues%2F25&data=04%7C01%7C%7C47082ece18bd4f49c70e08d89aeaf313%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637429681172675698%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5CFgQdpo%2BkvJH9BtJvYtbrY4ckq3wu9nPcQpd03AZEU%3D&reserved=0>, or unsubscribe<https://eur05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACOK5L6VSRMCF6HDAJCK34TSTUYDHANCNFSM4UQ6JPJQ&data=04%7C01%7C%7C47082ece18bd4f49c70e08d89aeaf313%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637429681172685691%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yqZdluKrKpPiTzcs1ObE4QStvFnBR2ubN04ECgLU%2BbE%3D&reserved=0>.

[statsgeeknz]

Hi guys. Maybe due to this (from betfair 18th Nov), which killed our code around the start of Dec:

"Following this announcement from Feb 2019, we are now looking to remove TLS 1.0 support across all other API endpoints. Specifically, any request to the API that isn’t a Login.

Based on our analysis we can see that you are still making requests to the Betfair API using TLS 1.0 from IP address XXXX

We’d like to remove support for TLS 1.0 by Monday 30th November so are notifying you of this so you can make the necessary changes before then."

We've (well, BC, not me) forked this and will send a pull request with changes pretty soon. Might be some other issue you're having though.

[statsgeeknz]

@JJHeathview for the record - sort of yes, so we're getting rid of RCurl in favour of httr. I expect B will have something to say at some point.

[phillc73]

Apologies, for the delayed reply, I haven't used this code for such a long time now, so it isn't getting any love.

At one point I did start migrating from RCurl to httr. Then I decided actually, I didn't want to buy into the Tidyverse (yep, Datatable rather than dplyr for me), and was thinking of migrating back to RCurl. In short, the chances of me migrating everything to httr is slim. I'd rather force use of newer RCurl and TSL > 1.0.

Have a look at this StackOverflow post covering determining the SSL Version of your RCurl and also how to enforce later versions.

E.g.

RCurl::curlVersion()$ssl_version

Some functions, such as listCountries(), listClearedOrder() and listMarketPandL() were migrated to httr, so it might be worth checking if they still work, when listCompetitions(), still on RCurl, doesn't.

If someone would like to submit a PR to cover forcing newer versions of TLS > 1.0 with RCurl, I'd be very pleased to merge it.

Also, having said that, if someone keen and actively using this code would like to be added as a maintainer of this repository I'd also be happy to discuss.

[gilesdoy]

Thanks all for the suggestions. I'm a total newbie at all of this so not 100% sure I've completely got this right. But taking listCompetitions() as an example, is it as "simple" as converting the code
as.list(jsonlite::fromJSON( RCurl::postForm(
into its httr equivalent
httr::content( httr::POST(
?

[jackkelleher]

@gilesdoy had you any success with your proposed method?

[gilesdoy]

It does work, yes, if you recode the relevant parts of listMarketBook, listCompetitions, listMarketCatalogue and listMarketTypes into the httr structure. Not been able to turn that into a new/separate package though

[phillc73]

If you submit a pull request here I will do my best to merge it. If you're not familiar with that, simply paste the enhanced functions into Issue comments and I will integrate them.

[roseypro]

Hi all,
I have attempted to update OpenSSL to version1.1.1a, however RCurl doesn't seem to recognise it. When using RCurl::curlVersion() it shows the version as OpenSSL/1.0.0o.

When I check the OpenSSL version of the curl package using curl::curl_version() it does recognise it. Presumably this means libcurl is pointed at the updated OpenSSL version. Is there a way to point RCurl to the updated OpenSSL?

If not, will try and convert to httr.

Thx, Chris

[Soccerama]

Has anyone cracked this yet? I now have working copies on my own machine and would be prepared to try a pull request if there is nothing currently underway. (I've not done this before).

[Soccerama]

Good point - I did it by not using httr. I'm now trying to get httr work. @Soccerama How did you manage to get them working? This is the code I devised to try and transfer from RCurl to httr but I'm being thrown a deserialisation error. listMarketBook <- httr::content( httr::POST("https://api.betfair.com/exchange/betting/json-rpc/v1", httr::add_headers(headers), httr::content_type_json(), body = list(postfields = listMarketBookOps, ssl.verifypeer = sslVerify) ) )

[Soccerama]

I've spent some more time on this today and it looks like we are stuck on the same thing. No approach that I've tried gets the output into a dataframe as required. The solution that I have is borrowed from someone else and uses curl  so it is inconsistent with what is done in other functions which is not ideal.

[bradley-pearson6597]

@Soccerama managed to resolve it.

Here's the code that needs to be changed:
listMarketCatalogueOps <- jsonlite::toJSON(jsonlite::unbox(listMarketCatalogueOps))
product <- Sys.getenv("product")
token <- Sys.getenv("token")
listMarketCat <- httr::content(
httr::POST(url = "https://api.betfair.com/exchange/betting/json-rpc/v1",
config = httr::config(ssl_verifypeer = sslVerify),
body = listMarketCatalogueOps,
httr::add_headers(Accept = "application/json",
"X-Application" = product,
"X-Authentication" = token)), as = "text")
listMarketCat <- jsonlite::fromJSON(listMarketCat)
if (is.null(listMarketCat$error))
as.data.frame(listMarketCat$result)
else ({
if (!suppress)
warning("Error- See output for details")
as.data.frame(listMarketCat$error)
})

If you do it like this the output is a nice dataframe which you can then easily manipulate.

[Soccerama]

If you do it like this the output is a nice dataframe which you can then easily manipulate.

Bradley, when I copy your code across it doesn't seem to work for me.

Also, I found listCurrentOrders more difficult to work with - have you tried that one?

[Soccerama]

Okay, they're all working now! I will tidy it all up and aim to do a pull request in the next few days.

[bradley-pearson6597]

Brilliant! Is the output all good?

[phillc73]

Thanks everyone who contributed here and especially @Soccerama . I've merged the pull request. The code all looked like it made sense, but to be completely honest I didn't personally run the code against any API queries. I'm open to more feedback if anyone finds bugs.

There may be some versioning things to take care of. The DESCRIPTION file states httr (>= 1.4.0) but it may be that this needs to change.

[Soccerama]

Thanks. I did some testing but far from best practice so wouldn't be surprised if there were some issues.

How do we get a new release so it automatically rolls out to R?

Actually, I'm happy to go through and update the rest of the functions prior to doing a release. Shouldn't take too long.

[phillc73]

I never packaged this for CRAN, so you should be able to install direct from the GitHub Master Branch with:

# install.packages("devtools")
devtools::install_github("phillc73/abettor")
library("abettor")

Let me know if it doesn't work or you have problems.

[gilesdoy]

Really appreciate everyone's help in making this work over the last week, apologies I've not been able to help more. Thank you!