Comments (15)
My offer stands. I would pay $100 in BTC if this gets resolved and I would prefer to pay only to one. If @30SoD wants to pay you to merge from what I pay him, that's ok, too.
If you share a BTC address or better an LN address, I can also shoot you a tip for your project in general. I can't find a donate button.
from app-store-scraper.
If @facundoolano is ready with the deal, i can work on that. You can buymeacoffee @Giszmo instead, that would be better on my side
from app-store-scraper.
or may be can it be converted with the new request.js (Changed HTTP Library) as in utils folder of google-play-scrapper
from app-store-scraper.
We switched to axios before and it broke some features, don't have the link handy right now I think it was proxy. So we had to switch back
from app-store-scraper.
that would be so bad, let me see if i can work out some round around solution
from app-store-scraper.
is it the JSON String Response of the "body" which varies when changing from request to axios which is causing the problems. Let me try to get the lookup string from axios & compare ??
from app-store-scraper.
@facundoolano - Check this out, i have temporarily changed the app.js and common.js with the use of axios.
I am sending out just first JSON of the search at the moment. Can be worked out for array of elements as it was. This is just a starter
APP.JS
'use strict';
const common = require('./common');
const ratings = require('./ratings');
async function app (opts) {
if (!opts.id && !opts.appId) {
throw Error('Either id or appId is required');
}
const idField = opts.id ? 'id' : 'bundleId';
const idValue = opts.id || opts.appId;
const output = await common.lookup([idValue], idField, opts.country, opts.lang, opts.requestOptions)
.then((result) => {
if (!result) {
throw Error('App not found (404)');
}
if (opts.ratings) {
if (!opts.id) {
opts.id = result.id;
}
return ratings(opts).then((ratingsResult) => Object.assign({}, result, ratingsResult));
}
return result;
});
return output;
}
module.exports = app;
COMMON.JS
'use strict';
const axios = require('axios');
const debug = require('debug')('app-store-scraper');
const c = require('./constants');
function cleanApp (app) {
return {
id: app.trackId,
appId: app.bundleId,
title: app.trackName,
url: app.trackViewUrl,
description: app.description,
icon: app.artworkUrl512 || app.artworkUrl100 || app.artworkUrl60,
genres: app.genres,
genreIds: app.genreIds,
primaryGenre: app.primaryGenreName,
primaryGenreId: app.primaryGenreId,
contentRating: app.contentAdvisoryRating,
languages: app.languageCodesISO2A,
size: app.fileSizeBytes,
requiredOsVersion: app.minimumOsVersion,
released: app.releaseDate,
updated: app.currentVersionReleaseDate || app.releaseDate,
releaseNotes: app.releaseNotes,
version: app.version,
price: app.price,
currency: app.currency,
free: app.price === 0,
developerId: app.artistId,
developer: app.artistName,
developerUrl: app.artistViewUrl,
developerWebsite: app.sellerUrl,
score: app.averageUserRating,
reviews: app.userRatingCount,
currentVersionScore: app.averageUserRatingForCurrentVersion,
currentVersionReviews: app.userRatingCountForCurrentVersion,
screenshots: app.screenshotUrls,
ipadScreenshots: app.ipadScreenshotUrls,
appletvScreenshots: app.appletvScreenshotUrls,
supportedDevices: app.supportedDevices
};
}
const LOOKUP_URL = 'https://itunes.apple.com/lookup';
async function lookup (ids, idField, country, lang, requestOptions) {
idField = idField || 'id';
country = country || 'us';
const langParam = lang ? &lang=${lang}
: '';
const joinedIds = ids.join(',');
const url = ${LOOKUP_URL}?${idField}=${joinedIds}&country=${country}&entity=software${langParam}
;
const appData = axios.get(url).then(res => {
//Temporarily Sending First Data OUT (Directly as a JSON)
const app = res.data.results[0];
if ((app.wrapperType === 'undefined') || (app.wrapperType === 'software')) {
return app;
}
}).then(res => {
const app = cleanApp(res);
return app;
});
return appData;
}
function storeId (countryCode) {
const markets = c.markets;
const defaultStore = '143441';
return (countryCode && markets[countryCode.toUpperCase()]) || defaultStore;
}
module.exports = { cleanApp, lookup, storeId };
from app-store-scraper.
@facundoolano Would it be possible to migrate from request to got? (same as google-store-scraper)
from app-store-scraper.
from app-store-scraper.
Any chance this could be re-visited?
from app-store-scraper.
i could re-write the code removing the request module, if you guys want to, gimme a couple of days for it
from app-store-scraper.
If Facundo merges it, I would offer you $100 via BTC.
from app-store-scraper.
You'll have to pay me to merge then.
from app-store-scraper.
@facundoolano I really appreciate your efforts to date, but do you plan to continue supporting this repository or are you hoping that someone forks it and takes over? Thanks
from app-store-scraper.
@30SoD how do I buy you a coffee without a card payment? I'm allergic to these data harvesters that regularly leak the data.
from app-store-scraper.
Related Issues (20)
- Request for similar updated definition
- Remove deprecated request dependency HOT 1
- Downloading the data from app store without node js HOT 2
- Crash when an invalid language is provided HOT 1
- List not returning ranks for the following category:
- Getting information about In-App purchases HOT 4
- Ramda ReDoS vulnerability, dependency upgrade needed HOT 1
- The graphic and design category is missing from lists?
- Latest reviews
- returning the number of apps for a specific search term
- After some hundreds of successful calls, all or most fail HOT 2
- Getting some debug output instead of actual output HOT 1
- Reviews missing 'updated' property
- suggest url is not working HOT 2
- Discrepancy between iPhone App Store Search Results and app-store-scraper Search Results HOT 8
- npm package not updated HOT 1
- Error while installing app_store_scraper on Google Colab Notebook
- Similar stopped working
- Browser API (Discussion)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from app-store-scraper.