We're updating the issue view to help you get more done. 

Fixing Full Block Reports

Description

We are using has based mechanism to match the state of the datanodes and the metadata. In our solution false positive matches are possible. To overcome this we propose to do infrequent full block reports, that is, we ignore the hashes even if they match and check every block. By default this mechanism will be used after a certain number of hash based block reports from a particular datanode has been processed. For example if we have processed 100 (configurable number) block reports from datanode 'A' then that 101th block report from this datanode will ignore the all the hashes even if they match and do a complete block report.

Status

Assignee

Unassigned

Reporter

Salman Niazi

Labels

None

Priority

Medium