Skip to content

TVB-2430: Migration script from old to new storage version

Marmaduke Woodman requested to merge TVB-2430 into master

Created by: VinczeRobert

Please ignore the code ugliness of the code in the database migration script for now and the potential duplicate code in the h5 one, I will deal with those soon, but first I wanted to see the whole picture of the idea we discussed last Friday.

In the database migration code I will only migrate the tables that CAN'T BE CREATED FROM H5 (like Users, HasTraitsIndex and a few more).

Then when we reach the H5 migration code you will see that it does it a new thread and it loops over the datatypes in the database. I tried to migrate the HasTraitsIndex and DataTypes tables first, but it gives errors, it doesn't work (I am talking about line 200 in files_update_manager in master). So I thought to get all files with the H5 extension from the storage folder. There might be a better way to do this, but I didn't find one so far.

So I will solve the problems in this order in the following days:

  1. Find everything in the database that can't be created from the h5 files and extract/migrate the necessary information. I will write a list with these.
  2. Migrate the few H5 files that were left and extract duplicate code in functions. Also write down if there are some H5 files that I can't optain (like TractsH5).
  3. Recreate the DB from the H5 files + the extracted information at Step 1.
  4. Optimize solution (probably I will get some suggestions on those but I won't wait just for that).
  5. Testing.

Merge request reports