Update Hopsworks settings in cluster definition breaks flyway migrate


I theorized the following bug:

  • Install Hopsworks with the default values for the configuration ( which are written in the `Variables` table)

  • The cookbook will template the sql file with the default values and current services ips

  • Flyway will apply the sql files and store the checksums in the database

  • Do an upgrade using Karamel and change the value of one of these parameter. (Let's say a service ip)

  • The cookbook will template the sql file with the new value

  • Flyway will compute the checksum

  • Checksum will not match the one stored in the database

  • Migration will fail

My proposed solution is the following:

We create a new file variables.sql which contains a list of `replace` commands which will take care of inserting the configuration parameters. This file will always be executed by chef and not by flyway.
In this way we will avoid checksums conflicts in case of parameter changes.

The problem is of course all the previous files that we cannot touch otherwise the checksum will not match. I still haven't figured a way around it.
We can potentially go through all the files and replace the placeholders with the default values. Problem is that some of these values are ip addresses, which are cluster dependent.
The alternative, is that we go through all the files and we remove all the inserts. This will require a manual upgrade for all the existing clusters.



Fabio Buso


Fabio Buso