sandwichfarm/nostr-watch

nostr registry and monitor web-client

@nostrwatch is in heavy development, everything below is presently early alpha. Legacy nostrwatch is only in maintenance mode and will only be edited to keep the site running smooth-ish until it can be fully replaced.

@nostrwatch

A monorepo with packages consisting of modules and daemons for tracking various relay datapoints. All packages are loosely coupled enabling independent, full-stack or hybrid implementations

nostrwatch is an OpenSats grant recipient

Packages

alpha legend
------------
+ = stable
$ = heavy development
@ = light development
% = very early stage
~ = experimentation stage, prototyping
^ = porting from legacy, pushed when buggy but bootable
? = not started, research phase
! = refactoring

Apps

• gui-web [?]: Web app for monitoring relays. Has two modes, first one leverages data propagated by daemons to history relay(s) to seed the local dataset and nocap for client-side processing. Consumes existing packages by using an lmdb adapter for IndexedDb in the browser. Will be rewritten from the ground up.
• status [?]: Status daemon for monitors; watches for their output and detects downtime.
• ...TBA

Modules

• nocap [$]: Successor to nostrwatch-js, an extensible module for tracking various datapoints on relays.
• publisher [$]: A module and daemonn that standardizes @nostrwatch events for data propagation via relays.
• ...TBA

Utilities

• utils [%]: General utilities and shared stateless functionality.
• logger [+]: A wrapper for logging implemented by deamons.

Daemons

• trawler [$]: A daemon with the single purpose of collating, sanitizing and basic classification of relays. Daemon can leverage rest-api.
• nocapd [$]: A daemon that persistently monitors relays and produces a rich dataset. Daemon can leverage rest-api.
• ...TBA

Cache

• nwcache [@]: An interface for lmdb which is used as a processing cache.

Templates

• history-relay [?]: A few configs for the new nostr.watch history relay. History relays store events for the nostr.watch datalayer.
• redis [%]: Convenience configuration that standardizes redis configuration for stack. Primarily used for development and eventually deployments. Redis is used for BullMQ

Derivatives

• nostrawl [@]: A package for trawling any number of nostr relays. Generalized logic from trawler. Combines nostr-fetch and queues, to make coalescing data from specific filters simple.
• nostr-geotags [+]: A package for generating event geo g tags for events, needed for NIP-66.

Philosophy

nostr.watch legacy has been using nostr as a data layer successfully since February 2023, less some ... ehem ... hiccups. When it comes to the gui, it's a poor user experience that resulted from technical debt, scope creep and inopportune but uniquely opportune timing. It has never had any database. It has run entirely off data from nostr. Relays are the database.

@nostrwatch:next will continue to employ the philosophy of nostr as a source of truth. However, this time lmdb will be used for local cache for performance and to produce richer datasets. Each daemon has it's own database, but daemons in a shared environment can (or can not) share the same database.

Contribution

Contribution during this stage would be difficult but I'm open to it. If you want to contribute in any way, the best place to start is to open a discussion and review the nostrwatch project tracker. Projects will grow as I migrate relevant items from planning that are presently in Notion to Github (~December).

1. I often rapidly prototype in javascript, but modules will be ported to Typescript by the respective package's beta.
2. Daemons with little need for type safety are likely to stay vanilla javascript unless it becomes a priority or someone takes initiative.
3. Discuss -> Propose -> Execute

Development

a CONTRIBUTE.md will exist somewhere down the road. Since it's early stage, many details are not yet established, but here some details that are:

1. Primary branch for development @nostrwatch:next is next. Legacy is on main and the workflows are still functional for patches and legacy maintenance headaches.
2. Branching model: TBD (Trunk Based Development), trunk branch is next for early stage. Early alpha there will be long-standing branches for packages that are not yet in next. Once @nostrwatch:next reaches beta next will become main and there will be trunk branch.
3. GH Actions will be used for CI/CD
4. Issues will be reserved for actionable items, such as bugs. Anything requiring discussion lives in discussion until it's promoted to an issue.
5. PRs should be opened as a draft. When the PR is ready to be published commits should be squashed.

Development

It is not recommended to use any of these packages for any reason at this time, except maybe out of curiousity or masochistic desire.

git clone https://github.com/sandwichfarm/nostr-watch.git
cd ./nostr-watch
yarn bootstrap

Testing

Testing suite is not yet implemented.

nostr-watch's Issues

Can't yarn build because nostr.watch is down

Nostr.watch was unstable so i tought i deploy my copy but hey... i got this error. ^^

$ yarn build
yarn run v1.22.15
$ yarn get:relays && yarn get:geo
$ node ./scripts/relays.js
FetchError: request to https://nostr.watch/relays.json failed, reason: connect ECONNREFUSED 46.101.217.80:443
at ClientRequest. (C:\user\nostr-watch\node_modules\cross-fetch\node_modules\node-fetch\lib\index.js:1491:11)
at ClientRequest.emit (events.js:375:28)
at TLSSocket.socketErrorListener (_http_client.js:475:9)
at TLSSocket.emit (events.js:375:28)
at emitErrorNT (internal/streams/destroy.js:106:8)
at emitErrorCloseNT (internal/streams/destroy.js:74:3)
at processTicksAndRejections (internal/process/task_queues.js:82:21) {
type: 'system',
errno: 'ECONNREFUSED',
code: 'ECONNREFUSED'
}

Deploy workflow

Using GHA for now...

• Check for package.json version change
• yarn docker
• SSH, docker pull and restart containers

Abililty to add relays via nostr

Would be nice to be able to add relays via nostr, for a variety of reasons.

1. Publish an event with a specific event format to the nostr.relay (when public)
2. nostr.relay bot calls inspector to test the relay to prevent abuse (connect only)
3. If passed, nostr.relay calls a workflow that appends the relay to relays.yaml

Require SSL

You can access nostr.watch using both HTTP and HTTPS, but HTTP is not secure.
Images are being served using HTTP but should be served with HTTPS.
Please redirect users to HTTPS.

[0.1.0] Mobile Compatibilty

Describe the bug
Mobile is presently forcing a desktop view.

To Reproduce
Visit on mobile

Expected behavior
Returns responsive pages

Screenshots

Smartphone (please complete the following information):
all

Additional context

Technical debt

• remove redundant code
• Clean up HTML (mixed code styles)
• implement shared methods library
• rescope css in main.sass

NIP-11 info sometimes not saved to local storage

This is due to onComplete callback not being called for identities (NIP-05) or info (NIP-11) by https://github.com/dskvr/nostr-relay-inspector. Requires update outside the scope of nostr-watch

While the info is displayed on the page, a race condition will determine whether or not it ends up in local storage.

See: dskvr/nostrwatch-js#8

Add additional groupings

• By connection status
• By country
• By continent
• By nip support

Will require much needed refactor to what was a quick solution.

Analysis page

• Readable Network Summary paragraph
• Continent Summary
  • Country Drill down
• NIP Support

Refresh counter broke on last update

A bug was introduced while enforcing 'preferences.refresh', likely a simple fix.

Page jump on load

Page jump on load likely relates to the lifecycle management of vue3-leafleft, can possibly be mitigated with CSS only.

Page Flash

Describe the bug
Window flashes browsing between ages.

To Reproduce
Click on a relay

Expected behavior
Does not flash

Extension support

Add ability to sign messages with nostr extensions

Signature providers (nos2x, alby, etc)

• Add relay
• Use nostr user data to check their personalized connection status
• follow relay administrator
• Add to relays

LN Extensions (Alby, etc)

• Tip relay

Responsive

Initial version was created with tables for convenience, however, it was known from the beginning this was not the best option. I was attempting to avoid committing into a specific datatable library early, which can often heavily influence the underlying logic and can add limitations to features (speaking from experience)

In addition to a tableless/responsive design, there is the need for the following:

• #28
• filters
• grouping (achieved with tabs for now)
• ordering

Possible Libraries

• https://datatables.net/blog/2022-06-22-vue - Simple, clean, no heavy styling. Missing some capabilities normally found in datatable solutions.

Geo Data

Generated at build-time so that no license is required and IPs of visitors aren't unwillingly leaked.

• Generate geo.yml (so that yaml-loader can be used, less dependencies) in pre-build
• Parse geo.yml at buildtime
• Populate GEO object and watch for matches to be state.complete on render.

Check for events from administrator key

Load events from administrator key, display in single component under something like "news"

Website isn't showing my relay name and pubkey

I edited my .nostr/settings.json file to show my relay's name and pubkey a couple of days ago, but I'm not seeing the changes. Is there somewhere else in my server's directory this file should be? This is the top section of the settings.json file.

root@nymsrelay:~/.nostr# cat settings.json

{
"info": {
"relay_url": "wss://nostr.nymsrelay.com",
"name": "Nym's Relay",
"description": "A nostr relay written in Typescript.",
"pubkey": "bcea2b98506d1d5dd2cc0455a402701e342c76d70f46e38739aadde77ccef3c9",

Add pages and routing

• home
• routing to individual relays (shows detail modal)

Permalinks are broken

Believe it has something to do with proxy_pass in serverside nginx config.

Calling any other URL but / will result in 404.

Page alerts about relay but doesn't provide details

The page "warns" about my relay.nostr.info that it returned five events when asking for one but I don't know which query this was or what these extra replies were. I'd like to fix this but ... miss info.

Since Jack joined nostr, my relay is basically dead, so there probably is an issue but details would be great.

Alternate row colors

Please alternate row colors so it's easier to read the table in big screens

[0.1.0] Safari/Brave only checks 10-12 relays successfully.

Describe the bug
When visiting next.nostr.watch in brave, ~180 relays are checked in under a second and fail, leaving a handful to pass.

To Reproduce
visit next.nostr.watch with brave

Expected behavior
Checks all relays.

Beta/Public Repo

[0.1.0] Crashes chrome in production build on Netlify

Describe the bug
The production build is crashing on netlify in Chrome. The same production build does not crash locally.

To Reproduce
Steps to reproduce the behavior:

1. https://next.nostr.watch
2. Crash

Expected behavior
Does not crash

Screenshots

Desktop (please complete the following information):

• MacOS Mojave
• Chome:latest

Smartphone (please complete the following information):
n/a

Additional context

Implement Pinia

State management is presently a nightmare. Since nostr.watch is and will always be a single page, client-side app, I need to implement pinia (successor to vuex).

Benefits

• Easier state management
• Easier feature development
• Predictable behaviors
• Will close many existing issues and make the rest easier to complete.

0.1.0 Preview

Welcome! The preview for nostr.watch 0.1.0 is live. It is being deployed from release/0.1

Please leave feedback in this issue, if you find a bug, please submit a new issue with "0.1.0" in the title.

Feedback at this stage is most helpful where related to:

1. UX
2. Performance
3. Client-side bugs

Known Issues:

• #155 A bit wonky on Mobile. Minor mistakes made while templating, nothing huge.
• #156
• #157

Information about changes in depth will be included in 0.1.0 release notes.

Screenshots:

[0.1.0] Map doesn't load in safari

Describe the bug
Map doesn't load in safari

To Reproduce
visit next.nostr.watch

Expected behavior
map loads on pageload

Navigation

• Github
• Submit Relay
• About

Stuck on loop fetching nostr.json

Describe the bug
If NIP-05 nostr.json fails, nostr.watch get's stuck fetching it over and over again.
Nothing else works after that happens.

To Reproduce
See the console logs for:
https://nostrex.fly.dev/.well-known/nostr.json

Expected behavior
nostr.watch should retry the failed nostr.json with an exponential backoff and bail after 3 to 5 tries.

Screenshots

Desktop (please complete the following information):

• OS: Mac OS
• Browser: Chrome

Additional context
Add any other context about the problem here.

Relay Operator Feed

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Show notes by relay operator

Describe alternatives you've considered
none

Depends on #166

Fix various display bugs

• Copy is broken
• Do not show copy tooltip for offline relays
• Misalignment of restricted type relay status indicators
• Missing tooltips for location and identity columns
• JSON modals are not z-indexed properly

Remove events from info column

They are presently being overwritten anyways during processing due to an oversight. These messages should be displayed on the relay single component.

Add openstreetmap attribution

Absolutely required, oversight.

Relay added twice

After last PR, 2 new entries added

• wss://nostr.hackerman.pro
• wss://nostr.hackerman.pro.

The latter one accidentaly I guess.

Thank you in advance!

Feature request: Relay uptime

Hi there,

thanks for this great project. Would be helpful to see the uptime of each relay next to the ping.

Keep up the good work!

Tip Relay Operators via LUD06

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
A clickable lighting icon in relay list, relay map pop up and relay single that prompts a lighting payment.

Describe alternatives you've considered
None

Depends on #166

Scope CSS

Presently cluttered in main.scss

Feature Request: Pinned Relays

I wish I could pin my favorite relays to the top and let the rest get sorted by the algorithm.

Relays never finish checking

Describe the bug
Relays get stuck and never finishing checking.

To Reproduce
Load the page and look at the top right.

Expected behavior
Any bad relays timeout automatically.

Multi-sorting for results

Purpose would be to sort relays without read/write but connection higher up on the list. Latency cannot be tested without read (connect takes ~1 ms). Presently only sorting with latency, so they are scattered throughout the list which is not readable or helpful.

Sort by latency and then the various checks.

Relay administrator suggestions

Detect common issues and display them on relay detailed page.

• Detect Cors support and providing information on how to resolve (NIP-?? [don't remember rn])
• Detect Default Configs and how to resolve
• Detect invalid parameters (such as invalid public keys) and how to resolve

Style Modals

Remove nip-11 column

• Move modal trigger to relay url
• Add copy icon instead of click relay url to copy

Feature Request: Average Read Latency

The instant read latency is affected by the current network conditions and is jittery.
An average read latency could be calculated perhaps using a sliding window (or just adding up previous reads), whenever we "check" with the relay again.

Nostr driven, write

Complete extensions integration

• write relay to kind 3

Daemon (separate repo)

• Write to new replaceable kind with basic status checks.

Add link to nostr.json static api

Presently, a static API is available at nostr.watch/nostr.json to retrieve relays.

Identify how to describe it.

A better API that removes offline nodes would likely require a backend, which will not begin until stable. There are alternatives to a backend

1. client-side dynamic api, send appropriate headers. Problem here is it is inefficient with a long timeout.
2. build-time filtering of offline nodes. Problem here is that the data will be frequently out of date or incorrect (build-time is during a planned maintenance)

Component Based Queue

Is your feature request related to a problem? Please describe.
Identified an efficient pattern for running tasks that doesn't require webworkers or extraneous work in queue logic. Piggybacks off existing task management.

Describe the solution you'd like

• Component conditionals ensure only one task is ran at a time.
• Each task uses the tasks store for getting and setting task status
• When a task completes by signaling tasks store the component will unload, and another task component will take over.

Describe alternatives you've considered

• store driven queue
• other existing queue solutions

Dark mode

Nostr driven, read

Integrate nostr.watch events for runtime loader.

• Publish nostr events from daemon that include connect, read, write and relay info (presently alpha, pushing to wss://nostr.sandwich.farm with event kind: 1001)
• Ability to disable data manually from nostr when there are network issues.
• Fallback to prebuild data when there are network issues.
• check latency client-side with legacy fallback for accurate results.

Improve testing procedure

The testing procedure is based on the original testing procedure made by @fiatjaf, and it has some issues

1. nostr relay will only write a event once (rightfully)
2. nostr relay will return the expected event
3. nostr relay will return true for write even if that's no longer true (for example on my relay, which is now whitelisted only, but the event was indeed published in the past, so it returns a false write)

Alternatively

1. Send fresh events from nostr.watch account (updates, etc, not spam) to the nostr.watch private relay and a pool of relays determined by the contents of relays.yaml
2. Have a nostr-bot on the private nostr.watch relay that triggers a workflow whenever nostr.watch relay (private) is updated by nostr.watch account, this workflow will populate events in a file and commit to source and push. (CD would be triggered on push)
3. The client-side tests will then test the most recent event that was published, and if it exists on the relay, read is true.

The result will be a drastically improved, albeit delayed, write testing methodology that doesn't spam relays unnecessarily or send repeated publish attempts on every pageload.

When #93 is implemented, can use a users events to test their ability to read/write to each relay for more precise results.

Other areas of improvement

• latency (#100)

Considerations

• There will need to be an info page that informs relays not to block nostr.watch events.
• nostr.watch account will need to rate limit the updates as to avoid spamming relays with needless information
• nostr.watch updates should be relevant, timely and semi-frequent (1-2 a week)

Dependencies

1. Will require updates to nostr-relay-inspector to accept custom events for testing