Comments (4)
Handy trick for finding all missing nodes in a hyperdb:
var hyperdb = require('hyperdb')
var fn = '21b2b9ff201b01e6081709d82e6b81a5cf3a68d2cd5f092d0ffec58772642892'
var db = hyperdb(fn, { valueEncoding: 'json' })
db.ready(function () {
for (var i=0; i < db._writers.length; i++) {
var feed = db._writers[i]._feed
for (var j=0; j < feed.length; j++) {
var has = feed.has(j)
if (!has) console.log('MISSING', i, j)
}
}
})
from cabal-cli.
I dug into this. Notes:
- a lot of messages from one particular feed are missing; none of the usual peers seem to have them
- those missing messages are causing hyperdb to stall out on
get
andput
s, because to do either hyperdb must traverse a subset of the nodes on a key prefix. It reaches a missing entry and gets stuck waiting forever for that node to be downloaded. Some nodes can become articulation points in the lookup graph, effectively severing reachability to lots of older messages as well. - I talked to @mafintosh about it and he thinks he has some small code changes that would alleviate this
from cabal-cli.
If we were using kappa-core, I think we could just have multifeed-index skip non-present nodes, and index them later. If we use unordered view modules like unordered-materialized-kv and the ilk, downloading them at a later time won't cause the views to become inconsistent.
from cabal-cli.
this has been resolved, but let's re-open this issue in case it reappears!
from cabal-cli.
Related Issues (20)
- Render bug: online nicks with duplicates bleed bold over to the next line HOT 2
- Out of heap space crash HOT 8
- Could not install - sodium-native related error HOT 4
- installation hangs on arch HOT 1
- Can cabal support chatting in emacs? HOT 1
- Peer IDs don't show for some users HOT 1
- can't install when using zsh-nvm HOT 2
- how do i get my key? HOT 1
- some channels make the ui scroll a bit
- Receiving a private message doesn't open a new pane for them in the sidebar
- Make display of incoming PM configurable HOT 3
- Include timestamps on search results
- Top titlebar does not recover / redraw properly after being narrowed and re-widened
- Cabal (terminal) fails to start: ERR_INVALID_ARG_TYPE (v12 node) HOT 6
- Cabal binaries: "No native build was found for platform=linux ..." HOT 2
- ERR_INVALID_ARG_TYPE(name, 'number', value HOT 1
- Display the alias name of cabals when in multi-cabal mode HOT 1
- Sync indicators on peer list
- Running two instances on same laptop HOT 1
- Does Cabal work in countries with restricted or censored Internet access? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cabal-cli.