RE: AioHiveBot, minor milestone: Integrated support for hive-engine

avatar

You are viewing a single comment's thread:

The reason for the high latency on my node is due to the fact you were testing in NL, and the node is hosted in west coast USA, takes a little bit for packets to travel that far.

Some advice: Can you throw some compression headers in at this spot: https://github.com/pibara/aiohivebot/blob/main/aiohivebot/core.py#L39 to make sure the data is compressed(unless httpx adds them in automatically, I'm not too sure with python, worth researching).

Also, rather than including the /contracts and /blockchain paths, you could actually put them in the method of requests like so:

[
  {
    "method": "contracts.find",
    "jsonrpc": "2.0",
    "params": {
      "contract": "nft",
      "table": "STARinstances",
      "query": {
        "account": "foxon",
        "_id": {
          "$gt": 0
        }
      },
      "limit": 1000,
      "offset": 7000,
      "indexes": []
    },
    "id": 1
  }
]

A post of that to just https://engine.rishipanthee.com will yield you with the response you were looking for. This makes it quite handy when using JSONRPC2.0 batches to send out multiple queries in the same http call. Not using this method meant that you can't send both a contracts and a blockchain rpc request in the same http call.

I haven't seen any query that might be using high offset(ex get all nfts owned by a user or get all holders of a token) in your codebase(unless the goal is for the users to do that themselves), but if you add them in, please don't use high offsets to do so, here's a message that I sent on discord:

"""
spent some time looking into whta caused my node to go down yesterday(I think, maybe thrusday, been a busy week). Anyways, looking thru my logs, I saw a ton of requets with really high offsets. It's possibly to rewrite these queries to make them more efficent, so please take some time to do so. Some advice, keep track of the last _id value recieved, and look for entries that have an id greator than that rather than using offsets. It helps a ton. So for example(some js code):

const lastOwnedId = 0
const nfts = await this.hiveEngine.queryContract({contract: "nft", table: token + "instances", query: {"account": this.props.username, "_id" : {"$gt" : lastOwnedId}}}, 0);
lastOwnedID = nfts[nfts.length - 1]._id
//(repeat until nfts.length is less than limit)

With an offset of 0, we can achieve the same result as increasing the offset while not eating up ram on the nodes.

If you have any apps that rely on high offsets to find everything, please update asap, I will eventually reduce the max offset on my node
"""

High offsets are bad for mongo since they eat up ram, so lets try and avoid that.

Overall, thanks for making this, I don't use python too much other than when needed, but good to see more choices in the ecosystem.



0
0
0.000
0 comments