Balancer slow and killing the database

Description

Running the balancer is very slow and may kill the database by overloading it.

The overloading comes from the fact that DatanodeStorageInfo.BLOCKITERATOR_BATCH_SIZE is too hight and can't be modified. This value is used when the balancer is calling getBlocks to get blocks on the datanode.

The slowness comes from the fact that getBlocks find the block to start from by going through all the blocks in the datanode which is very inefficient.

Status

Assignee

Gautier Berthou

Reporter

Gautier Berthou

Labels

None

Fix versions

Priority

High