TestBalancer regularly fails. This is due to two things
Calling scheduleAllBlockReport on the datanode does not work if it is called while the datanode is doing a block report.
Adding blocks in a block report is two slow.
We solve the first problem by making sure that when we call scheduleAllBlockReport the ongoing block report will not overwrite the time for the next block report.
We fix the second problem by grouping adding blocks inside transaction instead of adding the blocks one by one. On my setup for this particular test grouping blocks, addition within transaction reduced the adding time by a factor 10. This is still probably to slow for other purposes than this test so I will open another Jira to investigate more advance fixes.