Add unit tests for provenance part

Description

Currently, provenance is being tested using the Ruby tests in hopsworks. However, that complicates extending and testing code parts that touch the provenance code in HopsFS.

Here is an example of tests on metaEnabled directory

https://github.com/hopshadoop/hops/blob/master/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMetadataLog.java

Other issues:

  1. until we get dataset prov core set as subtree operation, I would need to be able to fall back to project prov core in case dataset is deleted and processed before the operations on the subtree files. If project is also deleted and processed before all files were processed, i would silently drop all operations since all archiving of prov ops is done within a project, when you delete a project you agree to delete all provenance information as well.

  2. remove xattr buffering - maintain only for prov core

Assignee

Alex Ormenisan

Reporter

Mahmoud Ismail

Labels

None

Fix versions

Priority

Medium
Configure