kylart/malscraper

Scrape everything you can from MyAnimeList.net

MalScraper

At the moment, MalScraper allows one to:

• Gather information about all the anime being released in a season.
• Gather anime-related news (include light-novels, manga, films...). 160 news available.
• Make an anime search (in 2 different ways!).
• Get different information for this anime.
• Get only the best result for an anime search.
• Get a list of an anime's episodes.
• Access the full official MyAnimeList API (includes search, add, update and delete from your user watch lists).

MalScraper is being developed mainly for KawAnime but anyone can use it for its own purpose.

Any contribution is welcomed.

Tables of content:

• Installation
• Use
• Methods

• • search.search()
• • getInfoFromName()
• • getInfoFromURL()
• • getResultsFromSearch()
• • getWatchListFromUser()
• • getSeason()
• • getNewsNoDetails()
• • getEpisodesList()
• • getReviewsList()
• • getRecommendationsList()
• • getStats()
• • getPictures()
• • getUser()

• Data models

• • Anime data model
• • Character data model
• • Staff data model
• • Search result data model
• • Seasonal release data model
• • Seasonal anime release data model
• • News data model

• Contributing
• License

Installation

npm install --save mal-scraper

Use

const malScraper = require('mal-scraper')

Methods

search.search()

Parameter	Type	Description
type	string	type of search (manga or anime)
opts	object	options for search (all keys are optional)

Usage example:

const malScraper = require('mal-scraper')
const search = malScraper.search

const type = 'anime'

// Helpers for types, genres and list you might need for your research
console.log(search.helpers)

search.search(type, {
  // All optionnals, but all values must be in their relative search.helpers.availableValues.
  maxResults: 100, // how many results at most (default: 50)
  has: 250, // If you already have results and just want what follows it, you can say it here. Allows pagination!

  term: 'Sakura', // search term
  type: 0, // 0-> none, else go check search.helpers.availableValues.type
  status: 0, // 0 -> none, else go check https://github.com/Kylart/MalScraper/blob/master/README.md#series-statuses-references or search.helpers.availableValues.status
  score: 0, // 0-> none, else go check search.helpers.availableValues.score
  producer: 0, // go check search.helpers.availableValue.p.<type>.value
  rating: 0, // 0-> none, else go check search.helpers.availableValues.r
  startDate: {
    day: 12,
    month: 2,
    year: 1990
  },
  endDate: {
    day: 12,
    month: 2,
    year: 2015
  },
  genreType: 0, // 0 for include genre list, 1 for exclude genre list
  genres: [1] // go check search.helpers.availableValues.genres.<type>.value
})
  .then(console.log)
  .catch(console.error)

Returns: A Anime search model or Manga search model object

getInfoFromName()

Parameter	Type	Description
Name	string	The name of the anime to search, the best match corresponding to that name will be returned
getBestMatch	Boolean	Whether you want to use match-sorter to find the best result or not (defaults to true)
type	string	The type, can be either manga or anime. Default is anime

Usage example:

const malScraper = require('mal-scraper')

const name = 'Sakura Trick'

malScraper.getInfoFromName(name)
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

// same as
malScraper.getInfoFromName(name, true)
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

malScraper.getInfoFromName(name, false)
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

// same as
malScraper.getInfoFromName(name, true, 'anime')
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

malScraper.getInfoFromName(name, false, 'anime')
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: A Anime data model object

getInfoFromURL()

This method is faster than getInfoFromName() as it only make one HTTP request

Parameter	Type	Description
URL	string	The URL to the anime

Usage example:

const malScraper = require('mal-scraper')

const url = 'https://myanimelist.net/anime/20047/Sakura_Trick'

malScraper.getInfoFromURL(url)
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: A Anime data model object (same as getInfoFromName())

getResultsFromSearch()

Parameter	Type	Description
query	string	The search query
type	string	The type, can be either manga or anime. Default is anime

Usage example:

const malScraper = require('mal-scraper')

const query = 'sakura'

malScraper.getResultsFromSearch(query, 'anime')
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: An array of a maximum length of 10 containing Search result data model objects

getSeason()

This method get the list of anime, OVAs, movies and ONAs released (or planned to be released) during the season of the specified year

Parameter	Optional	Type	Description
year	No	number	The year
season	No	string	The season, must be either spring, summer, fall or winter
type	Yes	string	The type, must be either TV, TVNew, TVCon, ONAs, OVAs, Specials or Movies

Usage example:

const malScraper = require('mal-scraper')

const year = 2017
const season = 'fall'

malScraper.getSeason(year, season)
  // `data` is an object containing the following keys: 'TV', 'TVNew', 'TVCon', 'OVAs', 'ONAs', 'Movies' and 'Specials'
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: A Seasonal release data model object

With type parameter:

Please note: 'TVNew' represents the 'New' anime for this season, whilst 'TVCon' represents the 'Continuing' anime in this season. 'TV' is simply an aggregate for both of these.

const malScraper = require('mal-scraper')

const year = 2017
const season = 'fall'
const type = 'TV' // Optional type parameter, if not specified will default to returning an object with all of possible type keys

malScraper.getSeason(year, season, type)
  // `data` is an array containing all the 'Seasonal anime release data objects' for the given type
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: A Seasonal anime release data model object

getWatchListFromUser()

From v2.11.6

Parameter	Type	Description
username	string	The name of the user
after	number	Useful to paginate. Is the number of results you want to start from. By default, MAL returns 300 entries only.
type	string	Optional, can be either anime or manga
status	number	Optional, Status in the user's watch list (completed, on-hold...)

Usage example:

const malScraper = require('mal-scraper')

const username = 'Kylart'
const after = 25
const type = 'anime' // can be either `anime` or `manga`
const status = 7 // All anime

// Get you an object containing all the entries with status, score... from this user's watch list
malScraper.getWatchListFromUser(username, after, type, status)
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

From v2.6.0

Parameter	Type	Description
username	string	The name of the user
after	number	Useful to paginate. Is the number of results you want to start from. By default, MAL returns 300 entries only.
type	string	Optional, can be either anime or manga

Usage example:

const malScraper = require('mal-scraper')

const username = 'Kylart'
const after = 25
const type = 'anime' // can be either `anime` or `manga`

// Get you an object containing all the entries with status, score... from this user's watch list
malScraper.getWatchListFromUser(username, after, type)
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: A User watch list data model object

v2.5.2 and before

Parameter	Type	Description
username	string	The name of the user
type	string	Optional, can be either anime or manga

Usage example:

const malScraper = require('mal-scraper')

const username = 'Kylart'
const type = 'anime' // can be either `anime` or `manga`

// Get you an object containing all the entries with status, score... from this user's watch list
malScraper.getWatchListFromUser(username, type)
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: A User watch list data model object

getNewsNoDetails()

Parameter	Type	Description
nbNews	number	The count of news you want to get, default is 160. Note that there is 20 news per page, so if you set it to 60 for example, it will result in 3 requests. You should be aware of that, as MyAnimeList will most likely rate-limit you if more than 35-40~ requests are done in a few seconds

Usage example:

const malScraper = require('mal-scraper')

const nbNews = 120

malScraper.getNewsNoDetails(nbNews)
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: An array of News data model objects

getEpisodesList()

Parameter	Type	Description
anime	object OR string	If an object, it must have the name and id property. If you only have the name and not the id, you may call the method with the name as a string, this will be slower but the id will be automatically fetched on the way
anime.name	string	The name of the anime
anime.id	number	The unique identifier of this anime

Usage example:

const malScraper = require('mal-scraper')

malScraper.getEpisodesList({
  name: 'Sakura Trick',
  id: 20047
})
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

//Alternatively, if you only have the name and not the id, you can let the method fetch the id on the way at the cost of being slower

const name = "Sakura Trick"

malScraper.getEpisodesList(name)
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: An array of Anime episodes data model objects

getReviewsList()

Parameter	Type	Description
anime	object	An object that must have the name and id property or just the name alone. If you only have the name and not the id, you may call the method with the name as a string, this will be slower but the id will be automatically fetched on the way
anime.name	string	The name of the anime
anime.id	number	The unique identifier of this anime
anime.limit	number	[optionnal] The number max of reviews to fetch - can be really long if omit
anime.skip	number	[optionnal] The number of reviews to skip

Usage example:

const malScraper = require('mal-scraper')

malScraper.getReviewsList({
  name: 'Sakura Trick',
  id: 20047,
  limit: 1,
  skip: 20
})
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

//Alternatively, if you only have the name and not the id, you can let the method fetch the id on the way at the cost of being slower

const name = "Sakura Trick"

malScraper.getReviewsList({
  name: 'Sakura Trick',
  limit: 1,
  skip: 20
})
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: An array of Anime reviews data model objects

getRecommendationsList()

Parameter	Type	Description
anime	object OR string	If an object, it must have the name and id property. If you only have the name and not the id, you may call the method with the name as a string, this will be slower but the id will be automatically fetched on the way
anime.name	string	The name of the anime
anime.id	number	The unique identifier of this anime

Usage example:

const malScraper = require('mal-scraper')

malScraper.getRecommendationsList({
  name: 'Sakura Trick',
  id: 20047
})
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

//Alternatively, if you only have the name and not the id, you can let the method fetch the id on the way at the cost of being slower

const name = "Sakura Trick"

malScraper.getRecommendationsList(name)
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: An array of Anime recommendations data model objects

getStats()

Parameter	Type	Description
anime	object OR string	If an object, it must have the name and id property. If you only have the name and not the id, you may call the method with the name as a string, this will be slower but the id will be automatically fetched on the way
anime.name	string	The name of the anime
anime.id	number	The unique identifier of this anime

const malScraper = require('mal-scraper')

malScraper.getStats({
  name: 'Ginga Eiyuu Densetsu',
  id: 820
})
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

//Alternatively, if you only have the name and not the id, you can let the method fetch the id on the way at the cost of being slower

const name = "Ginga Eiyuu Densetsu"

malScraper.getStats(name)
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: An array of Anime stats data model objects

getPictures()

Parameter	Type	Description
anime	object OR string	If an object, it must have the name and id property. If you only have the name and not the id, you may call the method with the name as a string, this will be slower but the id will be automatically fetched on the way
anime.name	string	The name of the anime
anime.id	number	The unique identifier of this anime

const malScraper = require('mal-scraper')

malScraper.getPictures({
  name: 'Ginga Eiyuu Densetsu',
  id: 820
})
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

//Alternatively, if you only have the name and not the id, you can let the method fetch the id on the way at the cost of being slower

const name = "Ginga Eiyuu Densetsu"

malScraper.getPictures(name)
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: An array of Anime pictures data model objects

getUser()

Parameter	Type	Description
name	string	The username of the user

  const malScraper = require('mal-scraper')

malScraper.getUser('Kame-nos')
  .then((data) => console.log(data))
  .catch((err) => console.log(err))

Returns: A User data model

Data models

Anime data model

You should treat all properties as possibly undefined/empty, the only guaranteed properties are title, type and id

Property	Type	Description
title	string	The title of the anime
synopsis	string	The synopsis of the anime
picture	string	The URL of the cover picture of the anime
characters	array	An array of Character data model objects
staff	array	An array of Staff data model objects
trailer	string	URL to the embedded video
englishTitle	string	The english title of the anime
synonyms	string	A list of synonyms of the anime title (other languages names, related ovas/movies/animes) separated by commas, like "Sakura Trick, Sakura Trap"
type	string	The type of the anime, can be either TV, OVA, Movie or Special
episodes	string	The number of aired episodes
status	string	The status of the anime (whether it is airing, finished...)
aired	string	The date from which the airing started to the one from which it ended, this property will be empty if one of the two dates is unknown
premiered	string	The date of when the anime has been premiered
broadcast	string	When the anime is broadcasted
volumes	string	The number of volumes of the novel
chapters	string	The numbers of chapters of the novel
published	string	The dates of publications of the novel
authors	string	The authors of the novel
serialization	string	The serialization of the novel
producers	array	An array of the anime producers
studios	array	An array of the anime producers
source	string	On what the anime is based on (e.g: based on a manga...)
genres	array	An array of the anime genres (Action, Slice of Life...)
duration	string	Average duration of an episode (or total duration if movie...)
rating	string	The rating of the anime (e.g: R18+..), see the List of possible ratings
score	string	The average score
scoreStats	string	By how many users this anime has been scored, like "scored by 255,693 users"
ranked	string	The rank of the anime
popularity	string	The popularity of the anime
members	string	How many users are members of the anime (have it on their list)
favorites	string	Count of how many users have this anime as favorite
id	number	The unique identifier of the anime
url	string	the URL to the page

List of possible ratings

Anime ratings can be either:

• G - All Ages
• PG - Children
• PG-13 - Teens 13 or older
• R - 17+ (violence & profanity)
• R+ - Mild Nudity
• Rx - Hentai

Anime search model

Property	Type	Description
thumbnail	string	Full url for anime thumbnail
url	string	Full url for anime page
video	string	full url of anime trailer video if any
shortDescription	string	Short description of the anime (or manga)
title	string	Anime title
type	string	Anime type
nbEps	string	Anime number of episodes
score	string	Anime score
startDate	string	Anime start date
endDate	string	Anime end date
members	string	Anime number of members
rating	string	Anime rating

Manga search model

Property	Type	Description
thumbnail	string	Full url for anime thumbnail
url	string	Full url for anime page
video	string	full url of anime trailer video if any
shortDescription	string	Short description of the anime (or manga)
title	string	Anime title
type	string	Anime type
score	string	Anime score
nbChapters	string	Number of chapters released so far
vols	string	Number of volumes released so far
startDate	string	Anime start date
endDate	string	Anime end date
members	string	Anime number of members

Staff data model

Property	Type	Description
link	string	Link to the MAL profile of this person
picture	string	Link to a picture of the person at the best possible size
name	string	Their name and surname, like Surname, Name
role	string	The role this person has/had in this anime (Director, Sound Director...)

Character data model

Property	Type	Description
link	string	Link to the MAL profile of this character
picture	string	Link to a picture of the character at the best possible size
name	string	Their name and surname, like Surname, Name
role	string	The role this person has/had in this anime (Main, Supporting...)
seiyuu	object	An object containing additional data about who dubbed this character
seiyuu.link	string	Link to the MAL profile of who dubbed this character
seiyuu.picture	string	Link to a picture of the seiyuu at the best possible size
seiyuu.name	string	Their name and surname, like Surname, Name

Search result data model

Property	Type	Description
id	number	The unique identifier of this result
type	string	The type of the result (e.g: anime...)
name	string	The title of the anime
url	string	The URL to the anime
image_url	string	URL of the image
thumbnail_url	string	URL of the thumbnail image
es_score	number	A number representing the accuracy of the result, where 1 is a perfect match and 0 a totally irrelevant one
payload	object	An object containing additional data about the anime
payload.media_type	string	The type of the anime, can be either TV, Movie, OVA or Special
payload.start_year	number	The year the airing of the anime started
payload.aired	string	The date from which the airing started to the one from which it ended
payload.score	string	The average score given to this anime
payload.status	string	The current status of the anime (whether it is still airing, finished...)

Seasonal release data model

Note: If nothing is found for the given date, the current year/season releases list will be returned

Property	Type	Description
TV	array	An array of Seasonal anime release data model objects
TVNew	array	An array of Seasonal anime release data model objects
TVCon	array	An array of Seasonal anime release data model objects
OVAs	array	An array of Seasonal anime release data model objects
ONAs	array	An array of Seasonal anime release data model objects
Movies	array	An array of Seasonal anime release data model objects
Specials	array	An array of Seasonal anime release data model objects

Seasonal anime release data model

Property	Type	Description
picture	string	Link to the picture of the anime
synopsis	string	The synopsis of the anime
licensor	string	The licensor
title	string	The name of the anime
link	string	The direct link to the anime page
genres	array	An array of strings which are the genres of this anime
producers	array	An array of strings which are the producers of this anime
fromType	string	From what this anime is based on/an adaptation of (Light novel, manga...)
nbEp	string	The number of aired episodes this anime has
releaseDate	string	When this anime has been released
score	string	The average score users have given to this anime

User watch list data model

v2.6.0

An array of User anime entry data model objects or User manga entry data model

v2.5.2 and before

Property	Type	Description
stats	object	A User stats data model object
lists	array	An array of User anime entry data model objects or User manga entry data model

User stats data model

Property	Type	Description
TV	string	Number of TV anime this user watched
OVA	string	Number of OVA anime this user watched
Movies	string	Number of Movies anime this user watched
Spcl	string	Number of special anime this user watched
ONA	string	Number of ONA anime this user watched
Days	string	Number of days spent in front of anime for this user
Eps	string	Number of eps watched by this user
MeanScore	string	Mean score given by this user
ScoreDev	string	Score deviation for this user

User anime entry data model

Property	Type	Description
status	integer	Status of the anime in the user's watch list (completed, on-hold...), see the Statuses references
score	integer	Score given by the user
tags	string	anime tags for this anime. Tags are separated by a comma
isRewatching	integer	Whther this user is rewatching this anime
numWatchedEpisodes:	integer	Number of episodes this user watched for this anime
animeTitle	string	The title of the anime
animeNumEpisodes	integer	How many episodes this anime has
animeAiringStatus	string	The status of the anime, see the Series statuses references
animeId	string	The unique identifier of this anime
animeStudios	string	Studios of this anime
animeLicensors	string	Who licensed this anime
animeSeason	string	???
hasEpisodeVideo	boolean	Whether episode information are available on MAL
hasPromotionVideo	boolean	Whether anime trailer is available on MAL
videoUrl	string	path to video url on MAL
animeUrl	string	path to anime url on MAL
animeImagePath	string	path to anime thumbnail url on MAL
isAddedToList	boolean	???
animeMediaTypeString	string	Type of this anime
animeMpaaRatingString	string	Rating of this anime
startDateString	string	When did this user start watching it
finishDateString	string	When did this user finish it
animeStartDateString	string	Start date of the anime following the format (MM-DD-YYYY)
animeEndDateString	string	End date of the anime following the format (MM-DD-YYYY)
daysString	string	???
storageString	string	Storage type for this anime (set by the user)
priorityString	string	Priority of this anime for the user

User manga entry data model

Property	Type	Description
myID	string	Deprecated
status	string	Status of the manga in the user's watch list (completed, on-hold...), see the Statuses references
score	string	The score the user has given to this manga
tags	string	The tags the user has given to this manga
isRereading	string	Whether the user is re-reading this manga or not, where 0 means not
nbReadChapters	string	Count of how many chapters of this manga the user has read
nbReadVolumes	string	Count of how many volumes of this manga the user has read
mangaTitle	string	The title of the manga
mangaNumChapters	string	Total count of chapters this manga has
mangaNumVolumes	string	Count of volumes this manga has
mangaPublishingStatus	string	The status of the manga, see the Series statuses references
mangaId	string	The unique identifier of this manga
mangaMagazines	string	Magazines where this manga airs
mangaUrl	string	Path to manga page
mangaImagePath	string	path to manga thumbnail
isAddedToList	boolean	???
mangaMediaTypeString	string	The type of the manga, see the Types references
startDateString	string	A mm-dd-yyyy format date of when the user started watching this manga
finishDateString	string	A mm-dd-yyyy format date of when the user finished watching this manga
mangaStartDateString	string	A mm-dd-yyyy format date of when the manga started
mangaEndDateString	string	A mm-dd-yyyy format date of when the manga ended
daysString	string	???
retailString	string	???
priorityString	string	Priority of this manga for the user

The types, statuses and series statuses aren't explicitly given by MyAnimeList, a number is given instead, here's the corresponding statuses/types according to their numbers

Types references

• 0: Unknown
• 1: TV | Manga
• 2: OVA | Novel
• 3: Movie | One-shot
• 4: Special | Doujinshi
• 5: ONA | Manhwha
• 6: Music | Manhua

Statuses references

• 1: Watching | Reading
• 2: Completed
• 3: On-hold
• 4: Dropped
• 6: Plan-to-watch | Plan-to-read

Series statuses references

• 1: Currently airing | Publishing
• 2: Finished airing | Finished
• 3: Not yet aired | Not yet published

News data model

Property	Type	Description
title	string	The title of the news
link	string	The link to the article
image	string	URL of the cover image of the article
text	string	A short preview of the news description
newsNumber	string	The unique identifier of the news

Anime reviews data model

Property	Type	Description
author	string	The name of the author
date	date	The date of the comment
seen	string	The number of episode seen
overall	number	The overall note of the anime
story	number	The story note of the anime
animation	number	The animation note of the anime
sound	number	The sound note of the anime
character	number	The character note of the anime
enjoyment	number	The enjoyment note of the anime
review	string	The complete review

Anime recommendations data model

Property	Type	Description
pictureImage	date	The link of the picture's anime recommended
animeLink	string	The link of the anime recommended
anime	number	The name of the anime recommended
mainRecommendation	number	The recommendation
author	string	The name of the author

Anime episodes data model

Property	Type	Description
epNumber	number	The episode number
aired	string	A "Jan 10, 2014" date like of when the episode has been aired
discussionLink	string	-
title	string	The title of the episode
japaneseTitle	string	The japanese title of the episode

Anime search results data model

Property	Type	Description
id	string	The unique identifier of this anime
title	string	The title of the anime
english	string	The english title of the anime
synonyms	string	A set of synonyms of this anime
episodes	string	The total count of aired episodes this anime has
score	string	The average score given by users to this anime
type	string	The type of the anime (TV, OVA...)
status	string	The status of the anime (Airing, Finished airing...)
start_date	string	A yyyy-mm-dd date format of when the anime started to be aired
end_date	string	A yyyy-mm-dd date format of when the anime finished
synopsis	string	The synopsis of the anime
image	string	URL to the cover image of the anime

Manga search results data model

Property	Type	Description
id	string	The unique identifier of this manga
title	string	The title of the manga
english	string	The english title of the manga
synonyms	string	A set of synonyms of this manga
chapters	string	The total count of chapters this manga has
volumes	string	The total count of volumes this manga has
score	string	The average score given by users to this manga
type	string	The type of the manga (Manga, Doujinshi...)
status	string	The status of the manga (Publishing, Finished...)
start_date	string	A yyyy-mm-dd date format of when the manga started publication
end_date	string	A yyyy-mm-dd date format of when the manga finished
synopsis	string	The synopsis of the manga
image	string	URL to the cover image of the manga

Anime stats data model

Property	Type	Description
watching	number	The total number of person who are watching the anime
completed	number	The total number of person who completed the anime
onHold	number	The total number of person who stop watching the anime but will continue later
dropped	number	The total number of person who stop watching the anime
planToWatch	number	The total number of person who plan to watch the anime
total	number	Total of stats
score10	number	The number of person ranking the anime with a 10/10
score9	number	The number of person ranking the anime with a 9/10
score8	number	The number of person ranking the anime with a 8/10
score7	number	The number of person ranking the anime with a 7/10
score6	number	The number of person ranking the anime with a 6/10
score5	number	The number of person ranking the anime with a 5/10
score4	number	The number of person ranking the anime with a 4/10
score3	number	The number of person ranking the anime with a 3/10
score2	number	The number of person ranking the anime with a 2/10
score1	number	The number of person ranking the anime with a 1/10

Anime pictures data model

Property	Type	Description
imageLink	number	The link of the image

Contributing

1. Fork it!
2. Create your feature branch: git checkout -b my-new-feature
3. Commit your changes: git commit -am 'Add some feature'
4. Push to the branch: git push origin my-new-feature
5. Submit a pull request.

License

MIT License

Copyright (c) Kylart

malscraper's Issues

No get all anime function/method

I think there's no getAllAnimes function or anything similar that will get all animes that's in the API.

undefined in everything

so, my data is returning undefined when i put like data.title and etc

print

Can't use MalScraper because anything results 403 forbidden

Hello there, recently i am having this problem. MalScraper won't give the result, instead it returns '403 forbidden'.

I am using glitch.com server, i tried to my different project but the result is still the same, I don't know it's just server-side error or same problem to anyone, i hope you can provide help, thanks.

Here is the log
https://drive.google.com/file/d/1LhsB--hf-DKGyBjKZg06kbwSF3NurkLm/view?usp=drivesdk

Cors problem on electron + webpack + vue

XMLHttpRequest cannot load https://myanimelist.net/search/prefix.json?type=anime&keyword=naruto. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:9080' is therefore not allowed access.

mal-scraper 2.2.1

electron 1.7.5

Support for episode titles

for example https://myanimelist.net/anime/31339/Drifters/episode

what i want to be in this api?

• Episode Title
• Aired

getRecommendationsList parameter object does not need name

malScraper.getRecommendationsList({ name: 'Sakura Trick', id: 20047 }) has the same result as
malScraper.getRecommendationsList({ name: 'anything', id: 20047 })
Looks like MAL doesn't care about the name. Ex https://myanimelist.net/anime/73/anything/userrecs works.

getRecommendationsList should be callable with just an id, no need to provide a name.

search maxResults not working

Although I'm really enjoying this API for my little project there is a tiny issue that causes a lot of stuff to go slower,

in the search function the option maxResults isn't functional. No matter what is put there (1, 250, 40) it will always return 50.

const malScraper = require('mal-scraper')
const search = malScraper.search

var anime = search.search('anime', {
    maxResults: 100,
    term: argument
 })
    .catch(console.error)
    .then((results) => console.log(results.length))

always returns 50. no matter what you set maxResults to.

remove leftover

is it possible to hide/remove leftover title for new season

[BUG] Synonyme & staffs

Hello,

I have been increasing the tests before starting a refactoring and I have noticed this :

const data = await getInfoFromURL('https://myanimelist.net/anime/30654/Ansatsu_Kyoushitsu_2nd_Season')

response error :

synonyms: 'Ansatsu Kyoushitsu Season 2, Ansatsu Kyoushitsu Final Season'

It should be an array, not a string.

Also, I have notice an error on the novel scrap with the staffs and characters fields that are identical.

How to get titles without leftovers

how to getting title for new season without leftover (TV Continuing). please take a look here
spring 2018
https://pastebin.com/fXbsrzQZ
summer 2018
i have marked some leftover
https://pastebin.com/tkKAy5wQ

what i'm trying to achieve here were getting title without leftover (TV Continuing)
could we scaping season from this link?
https://www.livechart.me/summer-2018/tv?leftovers=false
summer for season 2018 for year

False scoreStats

Hi, I found an error in the scoreStats's data : the number of users if printed twice.
For example : Dragon Ball Z is scored by 488778488,778 users. (We have 488,778 printed twice)
and it's the same for all animes.
I put some screenshots for you to see the issue.

I hope you will fix that quickly

Sincery yours

-LePtitMetalleux

Search.search() misplaced value in data model

Hello there, apparently there is some misplaced value in search.search() data model, here is the screenshot.

officialApi.search() method returns "Cannot read property 'entry' of undefined"

I suspect that the entire official API part is broken, as i noticed that at least some of the endpoints of the official API now returns 404 errors, and that the official API documentation itself returns a 404

Error search anime with type

If I add in the search parameters the key "type", it returns an incorrect result, example:

✔️Correct
Code:

var malScraper = require("mal-scraper")
malScraper.search.search("anime", {
                "term": "BanG Dream! 3rd Season"
            }).then((data) => console.log(data[0])).
                catch((err) => err);

Result:

{
  thumbnail: 'https://cdn.myanimelist.net/r/100x140/images/anime/1985/104767.jpg?s=a0e163e0c3ee85136e20d9ed15490b99',
  url: 'https://myanimelist.net/anime/37870/BanG_Dream_3rd_Season',
  video: 'https://myanimelist.net/anime/37870/BanG_Dream_3rd_Season/video',
  shortDescription: 'As the all-girl band Poppin’Party continues to rise in fame, they find a poster for an event at the legendary Nippon Budoukan. Now determined to participate in the "BanG 
Dream," Kasumi Toyama and the...read more.',
  title: 'BanG Dream! 3rd Season',
  type: 'TV',
  nbEps: '13',
  score: '8.09',
  startDate: '01-23-20',
  endDate: '04-23-20',
  members: '13,238',
  rating: 'PG-13'
}

❌ Error
Code:

var malScraper = require("mal-scraper")
malScraper.search.search("anime", {
                "term": "BanG Dream! 3rd Season",
                "type": 1
            }).then((data) => console.log(data[0])).
                catch((err) => err);

Result:

{
  thumbnail: 'https://cdn.myanimelist.net/r/100x140/images/anime/4/84177.jpg?s=a7c4462cf6f5458a8b28496a75863876',
  url: 'https://myanimelist.net/anime/25777/Shingeki_no_Kyojin_Season_2',
  video: 'https://myanimelist.net/anime/25777/Shingeki_no_Kyojin_Season_2/video',
  shortDescription: 'For centuries, humanity has been hunted by giant, mysterious predators known as the Titans. Three mighty walls—Wall Maria, Rose, and Sheena—provided peace and protection 
for humanity for over a hundre...read more.',
  title: 'Shingeki no Kyojin Season 2',
  type: 'TV',
  nbEps: '12',
  score: '8.40',
  startDate: '04-01-17',
  endDate: '06-17-17',
  members: '1,082,750',
  rating: 'R'
}

[DOCUMENTATION] Duplication of section

Hello,

You have a duplication of an anchor in the documentation. I am not sure how you wanna turn this. So I just lest you know.
Check for Anime reviews data model

This section appear twice in the Readme.md

high vulnerability

High Server-Side Request Forgery

Package axios

Patched in >=0.21.1

Dependency of mal-scraper

Path mal-scraper > axios

More info https://npmjs.com/advisories/1594

Data fetching incorrect

Some data in the anime JSON returns incorrect
some examples are "title" that return "Top"
and picture, that returns undefined

No data from .getInfoFromName(name)

title: '\n Top\n ',
synopsis: '',
picture: undefined,

[QUESTIONS] Refactoring and informations about files

Hello,

I have seen that in your directory src/search, you have a file name constants.js.
I was wondering if this file is exclusively for the directory search or for the whole application ?
I am asking that because there are duplicate constants like BASE_URL across multiple files and I was wondering if I can put everything here or I create a new one at the root of the src directory.

Thank you for your answer in advance.

Anime Search data model resulted wrong information

As you can see in the image, title, synopsis, and picture are not defined and wrong, this error probably comes from MAL, who may change something on the page. Please fix, thank you.

Timed out while running tests

Hello,

I have been interested by your scrapper for MAL but by running your test, I am facing this problem :
npm test

Timed out while running tests

After multiple try, I manage to see that it always appends on the same part of the code. By example, for the test checking that Naruto has 500 episode, you have a 5 recursive calls. You get all the pages one after the other. Same goes for Drifters. But Ava has a timeout error trigger if the connection is a bit too long.

Sorted search results

Is there a way to get the results of a search sorted by the total amount of people who have scored/watched it?

title and synopsis undefined

well i am using mal-scraper and it doesn't show the tittle of the anime also it doesn't show the synopsis

Install Error

npm ERR! code ENOSELF
npm ERR! Refusing to install package with name "mal-scraper" under a package
npm ERR! also called "mal-scraper". Did you name your project the same
npm ERR! as the dependency you're installing?
npm ERR!
npm ERR! For more information, see:
npm ERR! https://docs.npmjs.com/cli/install#limitations-of-npms-install-algorithm

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\Berthens\AppData\Roaming\npm-cache_logs\2021-02-24T20_04_35_359Z-debug.log

Missing package-lock.json

You should commit package-lock.json, this file keep accurate versions of installed dependencies and avoid some problems.

shortDescription

idk if this is an issue but the shortDescription is unfinished. Is there a way to get the full description?

Manga Details

Is there no way to retrieve manga details like getInfoFromName(name) for anime? Especially since the officialApi method is broken :(

getSeason returned an empty object

When I tryed getSeason('2020', 'summer') it returns an empty object in the array:

Bug

TypeError: Cannot read property 'url' of undefined
at C:\Users\7thghoul\node_modules\mal-scraper\src\info.js:222:33
at processTicksAndRejections (internal/process/task_queues.js:97:5)

Genres not parsed correctly

Example: Action, Adventure, Comedy, Super Power, Martial Arts, Shounen are parsed as

[
  'ActionAction',
  '    AdventureAdventure',
  '    ComedyComedy',
  '    Super PowerSuper Power',
  '    Martial ArtsMartial Arts',
  '    ShounenShounen'
]

Fix typo

Fix typo in repo's description.

Current Description

Scrap everything you can from MyAnimeList.net

Suggestion

Scrape everything you can from MyAnimeList.net

getInfoFromName don't work

Node 10.6.0
Electron 2.0.5

TypeError: match is not a function
    at getResultsFromSearch.then (info.js?f8a3:154)
    at <anonymous>

Error pagination

Returns an error when trying to paginate an advanced search, which doesn't have as many elements
Code:

malScraper.search.search("anime", {
                "term": "12-sai.: Kiss, Kirai, Suki",
                "type": 2,
                "has": 50
            }).then((data) => console.log(data)).
                catch((err) => console.log(err));

Error:

TypeError: Cannot read property 'length' of null
    at hasNext (c:\Users\gonfe\Documents\GitHub\pruebas\node_modules\mal-scraper\src\search\index.js:145:27)
    at c:\Users\gonfe\Documents\GitHub\pruebas\node_modules\mal-scraper\src\search\index.js:163:22
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Character info search

maybe I'm missing something, I wanted to know if there is any way we can find character info using their name just like we do to find anime malScraper.getInfoFromName(query)
if there is no way currently see if you can add that feature.

Idk if it's an issue. But it's bothering me.

When i use
.getinfofromname('Anohana')
If returns an irrevelant anime but when i use
.getresultsfromsearch('Anohana')
It returns exactly what i want in the first item in the array.. it's not only anohana, it's the same with many other animes as well. What do you think about it?
Shall we use getresultsfromsearch and then pick it's first name or url and use that value to search using getinfofromname or getinfofromurl?

Support

Is there any way to add Nsfw / safe word filter to this

malScraper.getInfoFromName()

Fetching user info in getWatchListFromUser

Hey! This is more of a question / feature request.. Is it possible to fetch user info like username, image_url and stuff like that in the getWatchListFromUser()? So it can be:

username: 'exampleUsername',
image_url: 'somelink',
otherStuff: 'As many as you want'
data: [/* This array contains the usual return */]

If that's not possible can we have a getUser or add a functionality to search for users in the search function.. I don't mind any work arounds and whatever you find suitable.

The way I use getWatchListFromUser is that people can see each other's lists but I cannot display their avatar and info upon fetching the list. I can only show the data.

Thank you in advance, I'm a huge fan of the work btw. Much love

Can't check credentials

What I'm trying to do:
I'm trying to check credentials of user trying to login

Error
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not Found</h1> <p>The requested URL /api/account/verify_credentials.xml was not found on this server.</p> <p>Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.</p> </body></html>

Source Code
`const api = new malScraper.officialApi({
username: username,
password: password,
});

  try {
    const res = await api.checkCredentials();
    console.log(res);
  } catch (e) {
    console.log(e);
  }`

.getWatchListFromUser() issues

This issue addresses 2 bugs i ran into with the getWatchListFromUser() method :

First:

TypeError: Cannot read property '0' of undefined at parseString (D:\Felix\chino\indev\node_modules\mal-scraper\src\watchList.js:76:38) at Parser.<anonymous> (D:\Felix\chino\indev\node_modules\xml2js\lib\parser.js:303:18) at emitOne (events.js:115:13) at Parser.emit (events.js:210:7) at SAXParser.onclosetag (D:\Felix\chino\indev\node_modules\xml2js\lib\parser.js:261:26) at emit (D:\Felix\chino\indev\node_modules\sax\lib\sax.js:624:35) at emitNode (D:\Felix\chino\indev\node_modules\sax\lib\sax.js:629:5) at closeTag (D:\Felix\chino\indev\node_modules\sax\lib\sax.js:889:7) at SAXParser.write (D:\Felix\chino\indev\node_modules\sax\lib\sax.js:1436:13) at Parser.exports.Parser.Parser.parseString (D:\Felix\chino\indev\node_modules\xml2js\lib\parser.js:322:31)
Line 76 of the watchList file, when it resolves the promise, it tries reading the first element
of mal.myinfo (which i guess is supposed to be an array), however mal.myinfo doesn't seem to be
defined for all users.

By that i mean that it doesn't work with some users, and work with some others. For reference:

The bug happen with the user Paradoxalcorp (https://myanimelist.net/profile/Paradoxcorp)
Works as expected with the user niputi (https://myanimelist.net/profile/niputi)

Second:

Within the stats object that is in the object returned by getWatchListFromUser(), the property name of the property containing the username is undefined instead of username or name (see the screenshot)

Next Release ?

Hello,

I would like to know when the next release is planned?

Thanks.

Cannot read property 'forEach' of undefined

Getting this error when I try to get information about an anime by name using the function:
getInfoFromName()

Error Stack:

Cannot read property 'forEach' of undefined
   at node_modules/mal-scraper/src/info.js:193:23
   at runMicrotasks (<anonymous>)
   at processTicksAndRejections (internal/process/task_queues.js:93:5)

I hope it will be fixed soon!

Error: A 'declare' modifier cannot be used in an already ambient context.

With mal-scraper 2.12.0 and typescript 5.0.2 :

It seems that the declare at https://github.com/Kylart/MalScraper/blob/dev/src/index.d.ts#L4
declare let search: Search;
need to be remove.

Without it, the transpilation is working

Update on .getInfoFromName()

Can you make the character array result in .getInfoFromName() grab from the characters in 'more characters' linked page on the main Mal anime page?

This is what the page displays:

from the looks of it you could also grab favourite count of the anime from this

error

const malScraper = require('mal-scraper')
/home/pi/node_modules/mal-scraper/src/info.js:152
.then(async (items) => {
^

SyntaxError: Unexpected token (

takes few seconds to fetch

Im trying to the api and takes around 10 seconds to fetch something. (ex: 2019 summer)
is there some kind of pre requisite for the api?

Replace Request

Request has been deprecated since 2020 and has recently been marked with a moderate security vulnerability as seen here.

I see only the Official API section of the package uses request, so I'm assuming it can be replaced with axios or some alternative?

@Kylart sorry to mention you but I feel this is a relatively important issue as it shows several warnings on install.

picture is undefined

picture property is undefined, happens when you try to get info from url and with all animes.

[Feature Request]: Add Japanese fields

It doesn't look like the Japanese name for the field is currently supported for Japanese names, as seen here: https://myanimelist.net/manga/23390/Shingeki_no_Kyojin
進撃の巨人

Is it possible to add this data? Is it possible to search by Japanese title?

.getInfoFromName() shows a max 10 characters

Even though all the anime I search for has much more over 10 characters the listing shows only 10. Is their a way I can go around this to scrap all the characters listed under the anime on the my anime list website?

Lack of information when searching for a manga

Hello:

I really have no idea if there is a template for bug reporting. My apologies.

---

The problem in question is searching for information from a manga using the search.seach() method.

Example of my code:

const malScraper = require('mal-scraper')

malScraper.getInfoFromURL("https://myanimelist.net/manga/21479/Sword_Art_Online").catch(err => undefined).then((data) => {
  console.log(data)
})

The expected answer is the result with all the information of the manga in question.

The result I receive:

{
  title: '',
  synopsis: '',
  picture: undefined,
  characters: [
    {
      link: 'https://myanimelist.net/character/36765/Kazuto_Kirigaya',
      picture: 'https://cdn.myanimelist.net/images/characters/7/204821.jpg',
      name: 'Kirigaya, Kazuto',
      role: 'Main',
      seiyuu: [Object]
    },
    {
      link: 'https://myanimelist.net/character/36828/Asuna_Yuuki',
      picture: 'https://cdn.myanimelist.net/images/characters/15/262053.jpg',
      name: 'Yuuki, Asuna',
      role: 'Main',
      seiyuu: [Object]
    },
    {
      link: 'https://myanimelist.net/character/55147/Shino_Asada',
      picture: 'https://cdn.myanimelist.net/images/characters/10/235939.jpg',
      name: 'Asada, Shino',
      role: 'Main',
      seiyuu: [Object]
    },      role: 'Main',
      seiyuu: [Object]
    },
    {
      link: 'https://myanimelist.net/character/75450/Alice_Zuberg',
      picture: 'https://cdn.myanimelist.net/images/characters/7/348874.jpg',
      name: 'Zuberg, Alice',
      role: 'Main',
      seiyuu: [Object]
    },
    {
      link: 'https://myanimelist.net/character/36831/Suguha_Kirigaya',
      picture: 'https://cdn.myanimelist.net/images/characters/9/192515.jpg',
      name: 'Kirigaya, Suguha',
      role: 'Main',
      seiyuu: [Object]
    },
    {
      link: 'https://myanimelist.net/character/64523/Yuuki_Konno',
      picture: 'https://cdn.myanimelist.net/images/characters/4/278171.jpg',
      name: 'Konno, Yuuki',
      role: 'Supporting',
      seiyuu: [Object]
    },{
      link: 'https://myanimelist.net/character/43892/Yui',
      picture: 'https://cdn.myanimelist.net/images/characters/15/264165.jpg',
      name: 'Yui',
      role: 'Supporting',
      seiyuu: [Object]
    },
    {
      link: 'https://myanimelist.net/character/106217/Bercouli_Synthesis_One',
      picture: 'https://cdn.myanimelist.net/images/characters/6/245565.jpg',
      name: 'Bercouli Synthesis One',
      role: 'Supporting',
      seiyuu: [Object]
    },
    {
      link: 'https://myanimelist.net/character/103297/Tiese_Shtolienen',
      picture: 'https://cdn.myanimelist.net/images/characters/10/239851.jpg',
      name: 'Shtolienen, Tiese',
      role: 'Supporting',
      seiyuu: [Object]
    }
  ],
  staff: [
    {
      link: 'https://myanimelist.net/character/36765/Kazuto_Kirigaya',
      picture: 'https://cdn.myanimelist.net/images/characters/7/204821.jpg',
      name: 'Kirigaya, Kazuto',
      role: 'Main'
    },{
      link: 'https://myanimelist.net/character/36828/Asuna_Yuuki',
      picture: 'https://cdn.myanimelist.net/images/characters/15/262053.jpg',
      name: 'Yuuki, Asuna',
      role: 'Main'
    },
    {
      link: 'https://myanimelist.net/character/55147/Shino_Asada',
      picture: 'https://cdn.myanimelist.net/images/characters/10/235939.jpg',
      name: 'Asada, Shino',
      role: 'Main'
    },
    {
      link: 'https://myanimelist.net/character/70899/Eugeo',
      picture: 'https://cdn.myanimelist.net/images/characters/4/367313.jpg',
      name: 'Eugeo',
      role: 'Main'
    },
    {  link: 'https://myanimelist.net/character/75450/Alice_Zuberg',
      picture: 'https://cdn.myanimelist.net/images/characters/7/348874.jpg',
      name: 'Zuberg, Alice',
      role: 'Main'
    },
    {
      link: 'https://myanimelist.net/character/36831/Suguha_Kirigaya',
      picture: 'https://cdn.myanimelist.net/images/characters/9/192515.jpg',
      name: 'Kirigaya, Suguha',
      role: 'Main'
    },
    {
      link: 'https://myanimelist.net/character/64523/Yuuki_Konno',
      picture: 'https://cdn.myanimelist.net/images/characters/4/278171.jpg',
      name: 'Konno, Yuuki',
      role: 'Supporting'
    },
    {
      link: 'https://myanimelist.net/character/43892/Yui',
      picture: 'https://cdn.myanimelist.net/images/characters/15/264165.jpg',
      name: 'Yui',
      role: 'Supporting'
    },
    {
      link: 'https://myanimelist.net/character/106217/Bercouli_Synthesis_One',
      picture: 'https://cdn.myanimelist.net/images/characters/6/245565.jpg',
      name: 'Bercouli Synthesis One',
      role: 'Supporting'
    },{
      link: 'https://myanimelist.net/character/106217/Bercouli_Synthesis_One',
      picture: 'https://cdn.myanimelist.net/images/characters/6/245565.jpg',
      name: 'Bercouli Synthesis One',
      role: 'Supporting'
    },
    {
      link: 'https://myanimelist.net/character/103297/Tiese_Shtolienen',
      picture: 'https://cdn.myanimelist.net/images/characters/10/239851.jpg',
      name: 'Shtolienen, Tiese',
      role: 'Supporting'
    }
  ],
  trailer: undefined,
  englishTitle: 'Sword Art Online',
  japaneseTitle: 'ソードアート・オンライン',
  synonyms: 'S.A.O, SAO',
  type: 'Novel',
  episodes: '',
  status: 'Publishing',
  aired: '',
  premiered: '',
  broadcast: '',
  producers: [ '' ],
  studios: [ '' ],
  source: '#490Popularity #168Members 47,694NovelBUNBUN (Art), Kawahara, Reki (Story)',
  genres: [ 'Action', 'Adventure', 'Fantasy', 'Game', 'Romance', 'Sci-Fi' ],
  duration: '',
  rating: '',
  score: '8.11',
  scoreStats: 'scored by 18,566 users',
  ranked: "#4902  2 based on the top manga page. Please note that 'R18+' titles are excluded",
  popularity: '#168',
  members: '47,694',
  favorites: '3,353',
  id: 21479
}

You can also see an error in the data shown, for example, the staff in charge of the study are the characters in the story themselves.
I have checked if it happens in other sleeves and indeed the error persists.

mal-scraper: v2.9.0
nodejs: v12.16.2
npm: v6.14.4