boly38 / botensky Goto Github PK

View Code? Open in Web Editor NEW

0.0 1.0 0.0 509 KB

BlueSky bot account that bring happiness to BlueSky users

Home Page: https://bsky.app/profile/botensky.bsky.social

License: MIT License

JavaScript 87.43% Shell 0.09% CSS 1.85% EJS 10.63%

bluesky-bot bot nodejs flower plantnet

botensky's Introduction

Hi there 👋

🔭 I’m currently working on LiveObjects, Creharmony.fr, @botEnSky and some other repositories (ex.).
📫 To reach me, open an issue on target repository, or else poke me on BlueSky @boly38.bsky.social or via my email
🤖 If (and only if) you're not a bot, then you could pick one image of this page to find how to retrieve my email ^^
⚡ Fun fact: do you know what Geocaching is ? if yes, then did you try geokrety.org ?
💬 Errors are part of development process, don't worry, ask, learn, retry !

botensky's People

Contributors

Watchers

botensky's Issues

Test must be autonomous and avoid useless delays

we need to simulate mute action to avoid to conflict with current bot mute configuration

end of test - dont suffer of auditlog safe delay (reduce it for tests)

Test must be autonomous

mute/unmute

bluesky agent and login rate limits

keep last login when possible

Notify on error, season 2

following #4

it seems that when Render put application in hibernation mode, there is no signal handler or else there is no possible notify to discord.
we need another hook to send/notify errors logs on issue

render app won't wake up (quickly) - 6min - just to trace

Following #9 we've moved bot triggers to Github Actions

today :

with manual trigger ( from github actions )
AND asking UI directly,

We've got no answer and didnt produce reply even after 5 min
https://github.com/boly38/botEnSky/actions/runs/9266694015/job/25491467534

on render side, no activity is logged :(

EDIT: GH action OK after 6min44 !!

11h16 GHAction trigger
11h20 render first logs
11h22 GHAction trigger ends with success

create `prod` branch that map & deploy main with version tag

use node-mongotools as example
update actions secrets and variables

Make Bluesky reply with link for hastags

doc : https://docs.bsky.app/docs/advanced-guides/posts#mentions-and-links

(and add lang)

replyTo 400

botEnSky is unable to reply

to this post: https://bsky.app/profile/neoresistant.bsky.social/post/3kvopsv5hwk2m
embedding this image : https://bs.plantnet.org/image/o/a1079db51c9eddcc35b184b759730d701db9bbac

25-06-2024 12:54:26.928 [info ] 🧾️PlantnetApiService - Pla@ntNet identify following image : https://cdn.bsky.app/img/feed_fullsize/plain/did:plc:umlxafzutkwwnbonfpz7iazr/bafkreid6fwibnbokwfzig5bvcmzm35wofiwby5plgutjbcb4gtooo22y2m@jpeg
25-06-2024 12:54:32.695 [error] 🧾️PlantnetCommonService - 🖥️ Plantnet  👤 REDACTED replyTo {
  "status": 400,
  "error": "InvalidRequest",
  "success": false,
  "headers": {
    "access-control-allow-origin": "*",
    "content-length": "123",
    "content-type": "application/json; charset=utf-8",
    "date": "Tue, 25 Jun 2024 10:54:32 GMT",
    "etag": "W/\"7b-5OYjOucrkx456vG6nIBiH/VZWIA\"",
    "keep-alive": "timeout=90",
    "ratelimit-limit": "5000",
    "ratelimit-policy": "5000;w=3600",
    "ratelimit-remaining": "4997",
    "ratelimit-reset": "1719316472",
    "strict-transport-security": "max-age=63072000",
    "vary": "Accept-Encoding",
    "x-powered-by": "Express"
  }
}
25-06-2024 12:54:32.696 [info ] 🧾️PlantnetCommonService - Unable to make bluesky embed of image https://bs.plantnet.org/image/o/a1079db51c9eddcc35b184b759730d701db9bbac, so keep it as text link: impossible de répondre au post
25-06-2024 12:54:32.911 [error] 🧾️PlantnetCommonService - 🖥️ Plantnet  👤 REDACTED replyTo {
  "status": 400,
  "error": "InvalidRequest",
  "success": false,
  "headers": {
    "access-control-allow-origin": "*",
    "content-length": "123",
    "content-type": "application/json; charset=utf-8",
    "date": "Tue, 25 Jun 2024 10:54:32 GMT",
    "etag": "W/\"7b-5OYjOucrkx456vG6nIBiH/VZWIA\"",
    "keep-alive": "timeout=90",
    "ratelimit-limit": "5000",
    "ratelimit-policy": "5000;w=3600",
    "ratelimit-remaining": "4994",
    "ratelimit-reset": "1719316472",
    "strict-transport-security": "max-age=63072000",
    "vary": "Accept-Encoding",
    "x-powered-by": "Express"
  }
}
25-06-2024 12:54:32.912 [error] 🧾️PlantnetCommonService - 🖥️ Plantnet  👤 REDACTED Impossible d'identifier l'image de https://bsky.app/profile/neoresistant.bsky.social/post/3kvopsv5hwk2m avec Plantnet : impossible de répondre au post

Improvement - mention bot to ask Pl@ntNet id.

Suppose you see a post (A) with a flower.

You want a Pl@ntNet identification for this flower.

reply to this post (rA) with "@Botensky" mention,
then the bot must detect this reply ( rA )
bot must do an (A's image) identification
bot must post result as reply to (rA)

cronjobs targetting render.com was not working : "output too large"/"sortie trop grande"

17h15 & 17h20 simulation job triggers

Échec (sortie trop grande)
=> not sure what that means
this two attempts were not seen from render.com backend log

no log in simulation time window / 05:27 is a manual check

add plantnet identification bluesky bot

Got null in answer when bioclip dont find flickr link

Example : https://bsky.app/profile/botensky.bsky.social/post/3kx3bdj6aui2y

Need

Check if a free api exist in order to do a bird identification plugin

need: simple call with image content or image url, and as result, need some details apout bird species identification with confident ratio
database must be open and free to use (& if possible with a community or company behind) and sustainable over time

Candidates

relevant candidates list:

1. BioClip repository & demo : need to deep test but really good candidate (unable to find condition of use for now) (suggested by James 🙏 )
1. a dataset of 535 species + 📦 mvp mymale repo
1. https://blog.roboflow.com/bird-detection-api/ : cf. test under

need to dig more list

Google Cloud Vision API 💲 seems not free / "1000unit/month" (from price) .. from demonstration, it seems powerfull but need to dig more
Microsoft Azure Computer Vision API - doc 💲 price price2 (free plan 50 labels/project) / quick start

(seems) 🟥 out-of-scope list

birdNet app => it's a (funny) android app that recognize bird from a given recorded sound range.
iNaturalist API doc => unable to find api to identify an image
Merlin Bird ID website => it's an app, seems there is no API, send mail to ask.
Picture Bird - Bird Identifier - android app only

💲 💲 NOT free

yeschat.ai (Bird Buddy) (price - starting 8€/month) - really accurate
nyckel.com : need an input image that focus on the bird for better id. based on kraggle model too. Using pre trained classifier is not free (price)

other references

reddit same question
database of species avibase

ui - show last bot activity

as logs are volatile, when render stops the app, the logs are lost.

it is frustrating to arrive on the bot ui with 0 usefull bot data

we need a list of recent bot replies/activities using external provider as persistent storage input :

recent #bot posts : bluesky api
recent bot news logs : logtail api (to be confirmed!)

implementation notices :

use a cache to avoid burst effect
display data freshness (cache) datetime

AskPlantnet - first try - cant view the reply using https://bsky.app/

After the first AskPlantNet attempt #33, a fix was done #44
and as result we have an identification score post for the post with bot mention's parent with reply to the post with bot mention as expected

But on https://bsky.app/

from identification score post I cant see the parent post
from the post with bot mention I cant see the bot rely (even if I have reply count to 1)

maybe something is missing in replyTo meta-data

bird questions : dont use second choice & embed image Bufffer issue

➕ embed image issue

PlantnetCommonService - Unable to make bluesky embed of image https://bs.plantnet.org/image/o/133db35bf883fbd454742d7e2c9ab421cb6115b1, so keep it as text link: Buffer.from is not a function

this is import story : import { Buffer } from 'buffer';

bioclip bad score with empty candidate

some news show a bioclip bad score with empty candidate

add error log alert

~~setup error sensor on log provider~~
add notifyService

`ERR_20240711145115` > bioclip identification results in 404

news:

14:51:15 | Unexpected error,please look for or report it on [issues](https://github.com/boly38/botEnSky/issues) - ERR_20240711145115

server side:

11-07-2024 14:51:11.145 [info ] 🧾️PlantnetApiService - Pla@ntNet identify following image : https://cdn.bsky.app/img/feed_fullsize/plain/did:plc:boctbwi57t7ls5vvrch2bj4b/bafkreianyjz4xc57yefpp6k47ma4ewf4wtlrkgqtc6aoohsonzubv5pe2u@jpeg
11-07-2024 14:51:15.492 [error] 🧾️PlantnetApiService - Pla@ntnet identify error (404) Not Found - details:{"statusCode":404,"error":"Not Found","message":"Species not found"}
11-07-2024 14:51:15.493 [error] 🧾️PluginsCommonService - 🖥️ Plantnet  👤 REDACTED Impossible d'identifier l'image de https://bsky.app/profile/vedadceric.bsky.social/post/3kwylxwjtxp2z avec Plantnet : Cannot destructure property 'doSimulate' of 'options' as it is undefined.
11-07-2024 14:51:15.493 [error] 🧾️ExpressServer - Error id:ERR_20240711145115 msg:Plantnet unexpected error stack:undefined
11-07-2024 14:51:15.574 [info ] 🧾️News 📢 - Unexpected error,please look for or report it on  <a href="https://github.com/boly38/botEnSky/issues">issues</a> - ERR_20240711145115

Yep we show that in the image used, there is no target :) so 404 is correct, the plugin MUST undertake this usecase (like Plantnet)

Flickr 524 error

https://bsky.app/profile/firandfern.bsky.social/post/3kynqozmpuk2x

We have 524 instead of link

Error on blsky login 500

On bluesky login issue 500, the bot dont catch the reject and the bot crash

check error handler

hook response: 502 bad gateway
server log

[2024-05-23T20:20:00.017] [INFO] BlueSkyService - login botensky.bsky.social@https://bsky.social
[2024-05-23T20:20:10.503] [ERROR] BlueSkyService - XRPCError: Internal Server Error
    at ServiceClient.call (/opt/render/project/src/node_modules/@atproto/xrpc/dist/client.js:106:23)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async BskyAgent.login (/opt/render/project/src/node_modules/@atproto/api/dist/agent.js:214:25) {
  status: 500,
  error: 'InternalServerError',
  success: false,
  (...)
  [2024-05-23T20:20:10.504] [WARN] BotService - plugin response status:undefined msg:Failed to log in — double check your credentials and try again.
	node:_http_server:352
    throw new ERR_HTTP_INVALID_STATUS_CODE(originalStatusCode);
          ^
RangeError [ERR_HTTP_INVALID_STATUS_CODE]: Invalid status code: undefined
    at ServerResponse.writeHead (node:_http_server:352:11)
    at ServerResponse._implicitHeader (node:_http_server:338:8)
    at write_ (node:_http_outgoing:945:9)
    at ServerResponse.end (node:_http_outgoing:1056:5)
    at ServerResponse.send (/opt/render/project/src/node_modules/express/lib/response.js:233:10)
    at ServerResponse.json (/opt/render/project/src/node_modules/express/lib/response.js:279:15)
    at file:///opt/render/project/src/src/services/ExpressServer.js:85:58
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  code: 'ERR_HTTP_INVALID_STATUS_CODE'

UI - add image thumbnail

UI - add image thumbnail for each relevant news logs

UI - make grey bad score or no result news entries

UI - make pages responsive

screen width is not friendly for mobile browsing experience

news cache evict dont work on fresh news

it seems the setTimeoutis not used correctly (NewsService)

plantnet - multiple search - avoid previous candidate post

Following #17

We need to avoid verifying a given candidate multiple times,
for example when a Pl@ntNet identification gives 404 no result or a result with bad score (<20%)

Suggest: we may try to see if we could use "mask this post" feature ?

Plantnet identify more than one image

when a candidate has more than one image, try to identify each image instead of trying first image only

when there is an issue on some image, skip this image
when there is a winner for one image, add a reply with image index
when there is more than one image as winner, add "thread" symbol on first reply, and stack one replyTo per image plantnet result

Bluesky protected post, bot was able to reply

Visible by connected users
https://bsky.app/profile/fukuneko-photo.bsky.social/post/3ku2ms3pjnk2m

Bot was able to reply
User cant reply
User cant view reply

EDIT:

(and Original Post Owner cant view reply too)
I posted a discussion about backend success response here

Add Pl@ntNet Question "#photography #flower"

add this to get more results

Plantnet - add id example image as embed post record

(context: following a good Pl@ntNet identification, a reply is done, example)
On bluesky the reply may include embed image, try to integrate it directly instead of a link

Pl@ntNet - impossible de répondre au post - replyTo status:400

on SIMULATION, plantnet identification is OK

02-06-2024 12:21:26.650 [info ] 🧾️BotService - 🖥️ Plantnet  👤 REDACTED_IPS plugin result Post:
	https://bsky.app/profile/neoresistant.bsky.social/post/3ktutqfpfcc2x 2024-06-01 18:16:38 by @Jean Gautier   🐝🐢🦊🕊️ (https://bsky.app/profile/neoresistant.bsky.social) --- #FleurisTonFil
Le très joli compagnon blanc (ou silène blanche)
#photographie #photography
	SIMULATION - Réponse prévue : Pl@ntNet identifie (à 61.65%) Silene latifolia (fam Caryophyllaceae) com. Silène blanc, Compagnon blanc, Lychnide blanche
#BeSPlantnet #IndentificationDePlantes

but on ACTION,

(news) the bluesky replyTo failed

12:22:59 | impossible de répondre au post

(server) with 400 status

02-06-2024 12:22:53.927 [info ] 🧾️Pl@ntNet - 🖥️ Plantnet  👤 REDACTED_IPS 1 candidate(s)
02-06-2024 12:22:53.928 [info ] 🧾️PlantnetService - Pla@ntNet identify following image : https://cdn.bsky.app/img/feed_fullsize/plain/did:plc:umlxafzutkwwnbonfpz7iazr/bafkreigdzlgx4jgoraqun3loh7t54k2nla4k7qdtik2nk5jlurcbvjne74@jpeg
02-06-2024 12:22:59.296 [error] 🧾️Pl@ntNet - 🖥️ Plantnet  👤 REDACTED_IPS replyTo {
  "status": 400,
  "error": "InvalidRequest",
  "success": false,
  "headers": {
    "access-control-allow-origin": "*",
    "content-length": "132",
    "content-type": "application/json; charset=utf-8",
    "date": "Sun, 02 Jun 2024 10:22:59 GMT",
    "etag": "W/\"84-8F/JCt3SgfypR1VYxro03Yu5m2I\"",
    "keep-alive": "timeout=90",
    "ratelimit-limit": "5000",
    "ratelimit-policy": "5000;w=3600",
    "ratelimit-remaining": "4997",
    "ratelimit-reset": "1717327379",
    "strict-transport-security": "max-age=63072000",
    "vary": "Accept-Encoding",
    "x-powered-by": "Express"
  }
}
02-06-2024 12:22:59.297 [warn ] 🧾️BotService - 🖥️ Plantnet  👤 REDACTED_IPS  plugin error: impossible de répondre au post

TODO:

add payload in server log error => we have no payload in bluesky at/proto client
probably linked to facets introduced with #25 (doc)
fix this 400 ;)

produce bot activity summary

create summary (per week / per month) of (most liked?)
check with discord (#4) to notify

regression - response with embed, got "undefined" instead of tags

example
https://bsky.app/profile/neoresistant.bsky.social/post/3ktksaociii2g

we have undefined instead of tags in replyTo text value

regression following #7

AskPlantnet replyTo parentOfCandidate instead of Candidate

in the following example: https://bsky.app/profile/rangedan.bsky.social/post/3ktxhb6yohk22

the bot produces a reply to the parent post

it was originally design to reply to candidate (mention of the bot)

impact:

next simulations keeps the same candidate again ❎
disable trigger until a fix is provided ✔️

summary : perf : init cache at startup

on render, at startup, the cache has no effect since more than 1 parallel query is done on backend
and there is many bluesky search for summary cache.

update logger to use betterstack

free plan (3d logs retention)
switch on winston logger & integration

plantnet - multiple search criterias

when there is no result for the first question, iterate over the rest of question's array until a post is found.

Bonus:

add botEnSky advanced search wanted improvement references close to the loop
add Plantnet dedicated wiki page to explain (lang:en) in details the plugin and add this public