Benchmarking Update: Better Node Scoring & Richer Metadata Coming Soon!
Hey everyone,
Just wanted to share a quick update on some behind-the-scenes improvements I've made to the hive-bench and engine-bench utilities – the tools that power the node performance data published to the @nectarflower (for Hive nodes) and @flowerengine (for Hive-Engine nodes) accounts.
Previously, the ranking of nodes was often based on simpler metrics or performance in individual tests. While useful, it didn't always perfectly reflect which nodes feel best in real-world usage.
The main improvement I've implemented is a new weighted scoring system. This system assigns different levels of importance (weights) to the various benchmark tests (like block retrieval, history lookups, API call speed, latency, config access, etc.). For example, latency and block retrieval speed might be weighted more heavily than how quickly a node responds to a simple config request, as those factors often have a bigger impact on user experience in apps and wallets.
The result is a single weighted_score for each node, where a higher score indicates better overall performance based on these weighted factors.
Richer Data in Metadata
This change also means the JSON data published to the @nectarflower and @flowerengine accounts will be more informative in the upcoming runs. You can expect to see:
- Nodes Sorted by Weighted Score: The primary list of nodes will now be ordered based on this new, more realistic performance score (best score first).
- Weighted Score Included: Each node entry will include its calculated
weighted_score. - Tests Completed Count: Each node will also show how many of the benchmark tests it successfully completed (e.g.,
tests_completed: 5/5or4/5). This gives a quick indicator of reliability across different test types. - Scoring Weights Published: The metadata will also include details about the specific weights used in the scoring calculation, for transparency.
Why Does This Matter?
The goal is to provide a more nuanced and useful picture of node performance. For developers using libraries like hive-nectar (which consume this data) or anyone manually selecting nodes, this weighted score and additional data should offer a better way to pick nodes that are not just technically working, but are likely to provide a better, faster experience for actual applications.
These updates (bumping the tools to v0.2.6) will be reflected in the benchmark runs and metadata updates going forward. Keep an eye on the @nectarflower and @flowerengine accounts!
As always,
Michael Garcia a.k.a. TheCrazyGM
Selling Nodes... Like only fans for coders.
hahaha...
"Send nodes" reminds me of a joke from my old college roomate:
"Click here to download your free printer!"
!PIMP
View more
Suggestion:
Link repo links to your posts. It will help people to follow more and explore.
That's an excellent and simple suggestion, I'll try to remember to do that from here on.
😇
Can nectarengine be downloadeded with pip yet? I'm working on trading bots for people to use on phones and Termux is a bit more touchy.... Beem it allows, I'm just checking for consistency in coding.
hive-nectar I did lower the python version to 3.10, nectarengine I have not, but I will soon if it would help you.
Termux doesn't like to go the long route to download it. If its not accessible via PIP on Termux it cant really be implemented easily to work on that OS correctly.
View more
I did get that. nectarengine still has the same requirements though. I can't use that at the moment.
View more
This?
{
"node": "https://api.hive-engine.com/rpc",
"SSCnodeVersion": "ivm-1.0.0",
"engine": true,
"token": {
"ok": true,
"count": 16,
"time": 3.0331857204437256,
"rank": 16
},
"contract": {
"ok": true,
"count": 13,
"time": 1.452458381652832,
"rank": 12
},
"account_history": {
"ok": true,
"count": 97,
"time": 30.00242781639099,
"rank": 1
},
"config": {
"ok": true,
"time": 0.15889835357666016,
"access_time": 0.000022649765014648438,
"rank": 5
},
"latency": {
"ok": true,
"min_latency": 0.12316370010375977,
"max_latency": 0.14050722122192383,
"avg_latency": 0.1305757522583008,
"time": 0.6528787612915039,
"rank": 15
},
"weighted_score": 54.12,
"tests_completed": 5
},
!BEER
!invest_vote
yes, the weighted score and the number of tests pass/failed was added to each of the reports. the very end of the meta also has a list of the test params and the weights used to calculate that score.
Thank you for another very useful improvement! This more-complete metric to measure node performance is so needed! 😁 🙏 💚 ✨ 🤙
This development is the gift that keeps on giving! Too bad we can't make a site button for this stuff!
!PIMP
!PIZZA
!PAKX
View or trade
PAKXtokens.How many? Link me please to some table, thank you. Prayers for Pakistan.
$PIZZA slices delivered:
@ecoinstant(4/20) tipped @thecrazygm
Come get MOONed!
@ervin-lemark denkt du hast ein Vote durch @investinthefutur verdient!
@ervin-lemark thinks you have earned a vote of @investinthefutur !
View or trade
BEER.Hey @thecrazygm, here is a little bit of
BEERfrom @ervin-lemark for you. Enjoy it!Learn how to earn FREE BEER each day by staking your
BEER.View or trade
BEER.Hey @thecrazygm, here is a little bit of
BEERfrom @ervin-lemark for you. Enjoy it!Learn how to earn FREE BEER each day by staking your
BEER.