marceloclp/calpheonjs

Calpheon.js is an interface to retrieve data about the BDO game by scraping BDOCodex.

Calpheon.js is an interface to retrieve data about the Black Desert Online (BDO) game by scraping BDOCodex. It's built using Typescript so you know exactly what is being returned.

Roadmap

This are the features that will be prioritized on future releases:

• Add support to all query types.
• Query using a search term.
• Add scraping support to:
  • Skills
  • Achievements
  • Nodes
  • Houses

Installation

npm install calpheonjs

Usage

import { Scrape, Item } from "calpheonjs";

const scrape = async (id: string, type: string) => {
    // Use Scrape if you don't know the entity type you are scraping.
    const entity = await Scrape(id, type);

    // Or use the proper scraper if you know what you are requesting.
    // Assume the entity you are requesting is an item.
    const item = await Item(id);

    console.log(item.data);
};
scrape('9213', 'item');

This is the output.

{
    "id": "9213",
    "icon": "/items/new_icon/03_etc/07_productmaterial/00009213.png",
    "name": "Beer",
    "name_alt": "맥주",
    "type": "item",
    "category": "Special Items",
    "description": "A mild alcoholic drink brewed from cereal grains",
    "prices": {
        "buy": 2150,
        "sell": 86
    },
    "grade": 1,
    "weight": 0.1,
    "effects": [
        "Worker Stamina Recovery +2",
        "(Use through the Worker Menu on the World Map)."
    ]
}

You can also query directly from the item, or using the provided Query function.

import { Item, Query, Queries } from "calpheonjs";

const scrape = async (id: string, type: string) => {
    const item = await Item(id);
    const recipes = await item.data.product_of_recipe();

    // Or you can use the Query function by passing the item id and the query type.
    // This is useful in case you don't care about the item data, only about the query results.
    const recipes = await Query('9213', Queries.Types.PRODUCT_IN_RECIPE);

    console.log(recipes.data);
};
scrape('9213');

This is the output.

[{
    "type": "recipe",
    "shortUrl": "/us/recipe/122/",
    "id": "122",
    "icon": "/items/new_icon/03_etc/07_productmaterial/00009213.png",
    "name": "Beer",
    "process": "Cooking",
    "skill_lvl": {
        "mastery": "Beginner",
        "lvl": 1
    },
    "exp": 400,
    "materials": [{
        "type": "material_group",
        "shortUrl": "/us/materialgroup/1/",
        "id": "1",
        "icon": "/items/new_icon/03_etc/07_productmaterial/00007005.png",
        "amount": 5
    }, {
        "type": "item",
        "shortUrl": "/us/item/9059/",
        "id": "9059",
        "icon": "/items/new_icon/03_etc/07_productmaterial/00009059.png",
        "amount": 6
    }, {
        "type": "item",
        "shortUrl": "/us/item/9002/",
        "id": "9002",
        "icon": "/items/new_icon/03_etc/07_productmaterial/00009002.png",
        "amount": 1
    }, {
        "type": "item",
        "shortUrl": "/us/item/9005/",
        "id": "9005",
        "icon": "/items/new_icon/03_etc/07_productmaterial/00009005.png",
        "amount": 2
    }],
    "products": [{
        "type": "item",
        "shortUrl": "/us/item/9213/",
        "id": "9213",
        "icon": "/items/new_icon/03_etc/07_productmaterial/00009213.png",
        "amount": 1
    }, {
        "type": "item",
        "shortUrl": "/us/item/9283/",
        "id": "9283",
        "icon": "/items/new_icon/03_etc/07_productmaterial/00009283.png",
        "amount": 1
    }]
}, ...]

Contributing

If you wish to help with this project, please follow the steps below. Any help is appreciated, but please stick to the same patterns that have been used accross the codebase.

1. Fork this repository.
2. Work on the features you want.
3. Make sure that all tests are passing by running npm run test or yarn test.
4. Create a PR when you are done.

Please follow this guide when naming your commits.

calpheonjs's Issues

V2 ts paths in dev environment not being compiled correctly

In the v2 branch. When you compile/build the project no errors are outputted. But, when you try to call the exported modules, they are not found. This is because the js complied exports try to use the '@' directory which is present when in the dev environment. I believe this is the issue. But not sure how to fix it since I lack typescript experience. Any help would be appreciated.

Cannot find module ./query

`[2020-11-29 16:17:35]: CMD [CMD] Bot Owner Ayeven (273097175452614666) ran command calpheon
[2020-11-29 16:17:35]: ERROR Unhandled rejection: Error: Cannot find module './query'
Require stack:

• /home/runner/ayevenbot/node_modules/calpheonjs/dist/index.js
• /home/runner/ayevenbot/commands/calpheon.js
• /home/runner/ayevenbot/modules/functions.js
• /home/runner/ayevenbot/index.js
  Error: Cannot find module './query'
  Require stack:
• /home/runner/ayevenbot/node_modules/calpheonjs/dist/index.js
• /home/runner/ayevenbot/commands/calpheon.js
• /home/runner/ayevenbot/modules/functions.js
• /home/runner/ayevenbot/index.js
  at Function.Module._resolveFilename (internal/modules/cjs/loader.js:965:15)
  at Function.Module._load (internal/modules/cjs/loader.js:841:27)
  at Module.require (internal/modules/cjs/loader.js:1025:19)
  at require (internal/modules/cjs/helpers.js:72:18)
  at Object. (/home/runner/ayevenbot/node_modules/calpheonjs/dist/index.js:14:14)
  at Module._compile (internal/modules/cjs/loader.js:1137:30)
  at Object.Module._extensions..js (internal/modules/cjs/loader.js:1157:10)
  at Module.load (internal/modules/cjs/loader.js:985:32)
  at Function.Module._load (internal/modules/cjs/loader.js:878:14)
  at Module.require (internal/modules/cjs/loader.js:1025:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
  '/home/runner/ayevenbot/node_modules/calpheonjs/dist/index.js',
  '/home/runner/ayevenbot/commands/calpheon.js',
  '/home/runner/ayevenbot/modules/functions.js',
  '/home/runner/ayevenbot/index.js'
  ]
  }`
  I'm not sure if this only happen if you using it on repl.it. I have not tested on development platform/own pc. I have not tried to manually upload the necessary folder to my repl.it project either. I have tried the older bdo-scrapper and it working on my repl.it

JSON output too long (?)

Attempting to fetch 715005 returns an error.

const itemFetch = await Item(itemID); // was also tried with Equipment
console.log(itemFetch);

SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at Equipment.parseEnchantmentArray (node_modules\calpheonjs\dist\scraper\builders\item\equipment.builder.js:110:34)
    at Equipment.get (node_modules\calpheonjs\dist\scraper\builders\item\equipment.builder.js:186:29)
    at Equipment.get (node_modules\calpheonjs\dist\scraper\builders\item\equipment.builder.js:178:25)
    at Equipment.<anonymous> (node_modules\calpheonjs\dist\scraper\builders\item\equipment.builder.js:320:139)
    at step (node_modules\calpheonjs\dist\scraper\builders\item\equipment.builder.js:76:23)
    at Object.next (node_modules\calpheonjs\dist\scraper\builders\item\equipment.builder.js:57:53)
    at fulfilled (node_modules\calpheonjs\dist\scraper\builders\item\equipment.builder.js:48:58)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)