Giter Site home page Giter Site logo

spencerwooo / substats Goto Github PK

View Code? Open in Web Editor NEW
798.0 7.0 55.0 4.66 MB

( `д′) how many followers do i have? how many!

Home Page: https://substats.swo.moe

License: MIT License

JavaScript 1.42% TypeScript 98.58%
cloudflare cloudflare-workers badge rss subscriber api api-service

substats's Introduction

substats

Get started · What's new? · Sponsoring

( `д′) how many followers do i have? how many!

Cloudflare Workers Version 2.0/substats Vercel

Get started

sub · stats /səb ˈ stats/

  • a serverless api for getting the number of followers of you in your favourite services

*Version 2.0 is still in beta, not all features are ported from 1.0. Check below for details 👇

Basic

https://api.swo.moe

You request:

GET /stats/:source/:key

I respond:

{
  source: string,
  key: string,
  failed: true | false,
  count: number | string  // Most often it's a number when source !== 'common'
}

Yep, it's that simple now. ;)

*Note that key needs to be url encoded, remember this if you are requesting the feedly, inoreader, or feedspub routes.

Building badges 🎫

Of course! And as a matter of fact, substats works quite well with shields.io's /dynamic route. All these badges below are dynamically generated with substat's data:

GitHub Telegram 微博 少数派 爱发电 即刻 Steam 知乎 哔哩哔哩 掘金 语雀

You can easily create your own badge with our badge builder at substats.swo.moe.

Badge builder screenshot

Advanced - the /common route 🍀

What if the source you are trying to use is not supported yet, but it's just a simple GET request? In this case, you can use the route:

GET /stats/common?endpoint=<url>&datapath=<path>

Such as:

GET /stats/common/?endpoint=https://api.genshin.dev/domains/cecilia-garden&datapath=rewards.0.details.2.mora

In this case, the endpoint is an API url:

https://api.genshin.dev/domains/cecilia-garden

The response this URL returns looks like:

{
  "name": "Cecilia Garden",
  "type": "Forgery",
  // ...
  "rewards": [
    {
      "details": [
        { /* ... */ },
        { /* ... */ },
        {
          "mora": 1125,
        },
      ]
    }
  ]
}

Hence, we provide the datapath as rewards.0.details.2.mora. (I specifically chose this data as it contains an array to demonstrate how to reference the value mora inside the array by index.)

Response from the endpoint provided by you is parsed with object-path, and the method for constructing a reference datapath to your value in the response is the same.

Try our /common route API URL builder here: substats.swo.moe/common.

Common route screenshot

Supported sources

  • afdian
  • bilibili
  • coolapk
  • feedly
  • feedspub
  • github
  • inoreader
  • instagram
  • jike
  • mastodon
  • medium
  • neteasemusic
  • reddit
  • sspai
  • steamgames
  • steamfriends
  • telegram
  • twitter
  • unsplash
  • weibo
  • wikipediazh
  • zhihu
  • juejin
  • yuque

logo_afdian logo_bilibili logo_coolapk logo_feedly logo_feedspub logo_github logo_inoreader logo_ins logo_jike logo_mastodon logo_medium logo_neteasemusic logo_reddit logo_sspai logo_steam logo_tg logo_twitter logo_unsplash logo_weibo logo_wikipedia logo_zhihu logo_juejin logo_yuque

What's new?

Yes, substats is now version v2.0-beta! Most of the updates are under-the-hood apart from API formats.

  • Refactored in TypeScript.
  • Updated to CloudFlare's module workers.
  • Worker is built with esbuild instead of webpack, extra fast!
  • Support for Newsblur has been deprecated (seems nobody uses it).
  • KV storages are now supported, some routes including instagram depends on this for storing cookies (wip).
  • Caching is ported to module workers in 2.0 and supported as always.
  • New documentation and query builder.

If you are looking for the multiple source and query functions in 2.0 - it's still under refactor, as itty-router does not parse multiple query parameters, blocking this feature here. You can still use the 1.0 route while we wait. README and documentation for v1.0 (deprecated).

Contributing

This is a monorepo managed by pnpm. Directory ./worker holds the Cloudflare Worker module, and ./docs is a React site for documentation (powered by Vite and Chakra UI). Check the README.md for both packages for details.

Sponsoring

Open-source is hard! If you happen to like this project and want me to keep going, please consider sponsoring me or providing a single donation! Thanks for all the love and support!

🧸 Please donate - 微信/支付宝 · Patreon · 爱发电

License

MIT

made with ❤️ by spencer woo

substats's People

Contributors

chawyehsu avatar dependabot[bot] avatar diygod avatar foskym avatar hardwaylinka avatar idealclover avatar kidonng avatar oldpanda avatar spencerwooo avatar tenkeyseven avatar yiukuenchu avatar zwithz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

substats's Issues

instagram queries always returns 0 followers

https://api.spencerwoo.com/substats/?source=instagram&queryKey=spencer_wooo

Result:

{
  "status": 200,
  "data": {
    "totalSubs": 0,
    "subsInEachSource": {
      "instagram": 0
    },
    "failedSources": {
      "substats-error": {}
    }
  }
}

I tried with my ins id and some others, the API always returns 0 followers. The handler code for instagram looks fine. Maybe there's something wrong with the backend. @spencerwooo

Instagram API strikes

Instagram has restricted using the unpublic user info API to retrieve a user's follower count (i.e., fetch from https://www.instagram.com/<username>/?__a=1 directly). By investigation, the restriction makes accessing the specific URL without a cookie redirect to the login page. Hence, the primary workaround is to extract a cookie if necessary and append the specific header with the cookie.

I would recommend taking advantage of KV. Explicit speaking, the cookie is extracted via a login API (inspired by instagram-user-feed) and stored in KV. If it is already stored, Substats should omit this step. Then, the user info API is still used but with enhanced logic. Since we are actually fighting against Instagram, let's name it "Plan Rebel". We need to know that this workaround has the following limits:

  • Highly unstable
    • This workaround requires no login confirmation, including two-factor authorisation and unusual login attempt confirmation. The former can be configured manually, while the latter may be impossible to avoid. This means that the account owner needs to tell Instagram that "it was me" tried to log in to my account every time the confirmation is prompted. Although the workaround has introduced a cookie caching mechanism to mitigate this limitation, it is still tremendously annoying for production.
    • Instagram can further restrict this workaround one day.
  • Limited resource: Cloudflare Worker free plan is used. KV is absolutely not unlimited. Yet I reckon it's sufficient.

Add support for Douyin 抖音

一些数据平台可以直接爬取抖音的账号粉丝数。应该可以实现数据抓取。

Thanks

CleanShot 2022-04-07 at 14 46 24

Some data platforms can directly crawl the number of followers of an account. Data crawling should be possible.

没有缓存模块?

好像每一次请求都是即时向上游服务发起请求抓取实时数据的?

Getting multiple query params

Hi there ✋

It's not sure that you know this way already, but I want to tell you. I know how to get multiple query params with ittry-router. Inside of the handler, we can get Request object. So, we can get multiple query params by getting url strings, passing into URLSearchParams, and using getAll() method. This is code:

import { Router } from 'itty-router'

const router = Router()

router.get('/', (req) => {
  const url = new URL(req.url)
  const search = new URLSearchParams(url.search)
  const params = search.getAll('foo')
  return new Response(JSON.stringify(params), {
    headers: {
      'Content-Type': 'application/json',
    },
  })
})

addEventListener('fetch', (event) => event.respondWith(router.handle(event.request)))

Run with Wrangler or Miniflare, and then access like this:

$ curl 'http://localhost:8787/?foo=hello&foo=morning&foo=night'
["hello","morning","night"]

You can get multi values.

Note:

You can also use Hono instead of ittry-router.

import { Hono } from 'hono'

const app = new Hono()

app.get('/', (c) => {
  const url = new URL(c.req.url)
  const search = new URLSearchParams(url.search)
  const params = search.getAll('foo')
  return c.json(params)
})

export default app

If it is annoying, feel free to close this issue.

Add support for multiple accounts on the same platform

Sometimes I need to track total subs of multiple accounts on a single platform. But the current query format

GET /?source={SOURCE}&queryKey={QUERY1}&source={SOURCE}&queryKey={QUERY2} ....

is not working properly. totalSubs is correct, but subsInEachSource only shows the last one. An example is here.

Requests for supporting new platforms / 有关支持新平台的请求

Submitting a request for supporting new platforms

If you want/wish Substats could support a new service/platform/website, please post comments directly under this issue.

如果你希望 Substats 可以支持一个新的平台、服务或网站,请直接在这一 issue 下面进行评论。你可以用「爱发电」来直接助力我对新服务与平台的支持!

爱发电


If you are willing to/going to directly contribute on supporting new services/platforms/websites, feel free to file a pull request after implementing said features.

如果你希望或准备直接提交一个支持新平台、网站或服务的功能,请直接向本项目进行 Pull Request,我们欢迎各种帮助。


Contribution guidelines are available here: How to contribute - Substats Docs

环境搭建与贡献指南的参考位于:How to contribute - Substats Docs

Platforms Milestone

(Future updates will all be posted here.)

Upcoming

  • CSDN (no obvious API)
  • SegmentFault (no obvious API)

Newly supported (Incomplete list)

  • 即刻(大 J 终于归来~)
  • Unsplash (Credits to @yiukuenchu)
  • 爱发电充电人次
  • 爱发电月收入

The Inoreader API is not working

The Inoreader API has not been working for about 3 months:

$ curl 'https://api.spencerwoo.com/substats/?source=inoreader&queryKey=https://blog.spencerwoo.com/posts/index.xml'
{"status":200,"data":{"totalSubs":0,"subsInEachSource":{"inoreader":0},"failedSources":{"inoreader":"RSS feed not found on Inoreader"}}}

index.js totalSubs 改为''比较好吧

async function fetchStats(sources, queryKey) {
  // function's returning value
  const fetchStatsRes = {
    totalSubs: '',
    subsInEachSource: {},
    failedSources: {},
  }

Zhihu API broken - 知乎API坏掉了

Zhihu badge broken due to the api stop working. See below.

image

Reason of error: api broken
https://api.swo.moe/stats/zhihu/li-yu-cheng-15
throws:
{"source":"zhihu","key":"li-yu-cheng-15","failed":true,"message":"请求参数异常,请升级客户端后重试。"}

Update plans for v3

  • Migrate worker to a more performant router - hono.
  • Resolve issues with particular routes (#48 #46 #45).
  • Rebuild site with Next.js and Tailwind CSS.

Add support for 少数派 metrics

Currently, there is follower counts support in Substats. It would be nice if it support metrics such as read counts, word count, etc.
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.