cyberbit / telem Goto Github PK
View Code? Open in Web Editor NEWTrivial ETL Engine for Minecraft
Home Page: http://telem.cyberbit.dev
License: Other
Trivial ETL Engine for Minecraft
Home Page: http://telem.cyberbit.dev
License: Other
Dump point-in-time metrics to filesystem with configurable filename and format (CSV, JSON). For multiplayer worlds, this could be used as a way to enable out-of-game resource leaderboards with a script that scrapes all the computer filesystems on a schedule.
Need to make adapters resilient against peripherals not existing when booting up. A good fix may be to execute peripheral.wrap(id)
before read()
or write()
calls if the stored peripheral is determined to be nil
.
Reported by juh9870
in Discord.
When a window (in this case a monitor) is resized, the plot area does not automatically pick up the change and becomes cut off.
Original monitor configuration:
Removing one column of monitors from the left:
A possible fix would involve checking the size of the assigned window at the start of every write call and updating internal structures. In addition to this, a function should be provided for triggering a layout refresh on the adapter manually.
AP Blocks:
Ideas:
Cool repository, should add: Ale32bit/RequireDB
This doesn't work:
:middleware(mw.custom(function(collection)
return telem.metricCollection({
sanity = -2
})
end))
EDIT: Now that Fluent exists, a direct implementation of middleware may be possible.
Things middleware should be able to do:
Two kinds of middleware:
The API to make this work is a little annoying, but could be a fun way to export for systems that don't support webhooks but can read from a URL.
nice
TBD
Adding individual files to the build script is bad. Auto-discovering files is good!
Some ideas:
Actually that's my only idea.
This is technically for further supporting Grafana, but should support anything that can ingest Prometheus.
When a secure modem output adapter tries to send a large (nearly 1,000 metrics) collection, the adapter crashes the backplane:
The solution may be to send multipart responses, where the requesting input adapter assembles the full collection when all parts are received. If the input adapter times out waiting for all parts, it can either stop listening to parts and assemble what it has received, or throw an error.
Alternatively, I can create a custom serde that is more efficient with keys and utilizes compression.
"POWAH"
someone, probably
I am yet unsure what versions have CC compatibility, in my testing on 1.18 I could not detect any peripherals.
Need abstraction layer for:
I did some brief testing, it looks like it is possible in theory, need a lot of abstraction but managed to get a janky version of inventory reading working.
Using Ocelot Desktop for emulation.
See https://docs.advanced-peripherals.de/peripherals/energy_detector/.
Offers transfer rate and transfer rate limit.
Tracking documentation for new adapters.
Links to https://telem.cyberbit.dev/assets/Telem_Mekanism_Panel.json, returns a 404
Some metric providers may interact with components that have a variety of metric types. Some of these metrics may be static, others may update once every minute, even more may be expected to update every tick. The current design treats all metrics the same, updating every metric during every cycle.
If an input adapter tightly integrates with its components (such as the Mekanism input adapters), it is fair to say there would be established knowledge on which metrics update "fast" and "slow".
The Backplane could be configured to support several cycle speeds, prioritizing certain inputs to be read more often than others in the same period of time. Input adapters themselves could define their APIs in such a way to divide the metrics into groups, like "fast", "slow", and "static". These groups would have default cycle times, but could be overridden by a user.
However, this introduces a challenge with how the Output Adapters should behave. Some output adapters may not benefit from accelerated update times, or cause confusing/corrupted results. Other outputs would greatly benefit, such as charts and other visual indicators.
This issue will serve as a place to record ideas during the planning of this feature.
Gather metrics about metrics!
Concepts:
Create these peripherals. These are supported directly by recent versions of Create: https://github.com/Creators-of-Create/Create/wiki/ComputerCraft-Integration
See CloudSolutions.
Basalt Graph and Plotter Line Chart record state over many cycles. It would be nice for that state to be somewhat recoverable after a program/computer crashes, or is unloaded/reloaded.
Continuing the work by @bananasov.
Need to evaluate input adapter capabilities and implement where possible. These will be utilizing the new Fluent query system established in the base Mekanism adapter. This is expected to significantly increase the size of the build if it is all included as part of the base install. If the need arises, I will start planning an adapter plugin system that can manage mod-specific features.
Targeting Bigger Reactors mod. Suggested by @scmcgowen.
Forces metrics to be "sticky", remaining in the backplane after their source disappears. If and when this occurs, the metric value is set to zero.
This would be intended for general storage inventories whose stored types may fluctuate often.
Concept:
backplane:addInput('custom_in_1', function ()
-- do stuff
-- can also return MetricCollection, this is sugar
return {
custom_metric_1 = 123,
custom_metric_2 = 4.999127
}
end)
backplane:addOutput('custom_out_1', function (metrics)
-- metrics = {
-- {
-- name = 'metric',
-- value = 123,
-- unit = 'unit',
-- adapter = 'adapter',
-- source = 'source'
-- },
-- ...
-- }
-- do stuff
end)
I'd love to get this on unicornpkg but the installer seems broken at this time :(
There needs to be some kind of in-game communication possible within Telem for coordinating systems across dimensions. Each computer attached to a peripheral can directly output its metrics to whatever local display it wants to, but having a home base that collects everything in one place is going to be important.
The input adapter would be a very specialized design as it needs to listen to events in order to collect them. The output would work like a normal output adapter, just screaming metrics into the void.
The input adapter sends requests to output adapters. The output adapters listen for requests.
Design notes:
adapter
? chest_1
and modem input adapter is storage_terminal
, the final adapter is storage_terminal:chest_1
cycle()
, every metric will have <adapter>:
prefixed to its adapter
property.Basalt added a Graph, now I want to use it.
these should all require filtering at load time
It doesn't work. Fix it.
In particular, the steam amount is missing.
Refresh the original four to ensure all metrics captured:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.