I tried to run hive-engine node in docker images some time ago. Worked like a charm, only problem was that it would take months or years to replay everything till blockchain head :)
Yes, restoring the database image is the bottleneck, a huge one. I hope too that some other approach will be created. Like continuing from the point in time aka from a specific block onward.
With the restore process itself, IOPS is the key. I don't think that parallelization has any impact. Disk speed is far more important.
With docker images and VMs you have additional layers between the process and the hardware that throttle it even further. I'll see how it will go with this setup above. And then when I move the whole VM to a NVMe disk :)
In my case the problem was with CPU. I ran it on some mobile AMD with multiple weak cores, but the process simply can't use more than one core. So it was processing like 20 blocks per second at best which was nothing in terms of iops.
I'm very interested to see how it will work for you
Yes, restoring the database image is the bottleneck, a huge one. I hope too that some other approach will be created. Like continuing from the point in time aka from a specific block onward.
With the restore process itself, IOPS is the key. I don't think that parallelization has any impact. Disk speed is far more important.
With docker images and VMs you have additional layers between the process and the hardware that throttle it even further. I'll see how it will go with this setup above. And then when I move the whole VM to a NVMe disk :)
Have a great day.
In my case the problem was with CPU. I ran it on some mobile AMD with multiple weak cores, but the process simply can't use more than one core. So it was processing like 20 blocks per second at best which was nothing in terms of iops.
I'm very interested to see how it will work for you
View more