[Scilab-users] h5mv / h5flush

classic Classic list List threaded Threaded
4 messages Options
paul francedixhuit paul francedixhuit
Reply | Threaded
Open this post in threaded view
|

[Scilab-users] h5mv / h5flush

Hi all

I'm trying to reorganize my hdf5 file by moving/renaming dozens of datasets, using mainly h5mv (loops are used to pass through all  groups and datasets)

My code is typically, where:
  • Source = the path+name of the original dataset
  • Destination= the path + the new dataset name
h5flush(h5);  
h5mv(h5, Source, h5, Destination);

Trouble I'm confront to:
  • It works only ounce and I do not understand why, even flushing hdf5 object
  • I can note that after h5mv, h5flush does not work anymore
  • hdf5 object becomes "invalide"
What am I doing wrong?

Thanks for any advice

Paul

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Antoine Monmayrant Antoine Monmayrant
Reply | Threaded
Open this post in threaded view
|

Re: h5mv / h5flush

Hello Paul,


This is not a solution, but just tests to assert that the problem does come from scilab and not from your datasets.

(1) Can you try to use command-line tool h5copy (that comes with the hdf5-tools pacakge on a Debian/Ubuntu Linux distribution) to do it outside of scilab?
If you succeed, that might be a bug in h5mv in scilab.
If you are under windows, there might be an equivalent tool, or you can use WSL if under Windows10.

(2) I get that you are running this code inside some sort of loop, right? Can you try to do it step-by-step on one of the file that fails? By step-by-step I mean either Ctrl+E line by line in your code or one command at a time in the console. This might help if it's a race condition bug. I faced similar issues when manipulating the graphic handle hierarchy within a loop.

Hope it helps,


Antoine

On 24/05/2020 12:22, paul francedixhuit wrote:
Hi all

I'm trying to reorganize my hdf5 file by moving/renaming dozens of datasets, using mainly h5mv (loops are used to pass through all  groups and datasets)

My code is typically, where:
  • Source = the path+name of the original dataset
  • Destination= the path + the new dataset name
h5flush(h5);  
h5mv(h5, Source, h5, Destination);

Trouble I'm confront to:
  • It works only ounce and I do not understand why, even flushing hdf5 object
  • I can note that after h5mv, h5flush does not work anymore
  • hdf5 object becomes "invalide"
What am I doing wrong?

Thanks for any advice

Paul

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
paul francedixhuit paul francedixhuit
Reply | Threaded
Open this post in threaded view
|

Re: h5mv / h5flush

Thanks Antoine for answering

The datasets work fine.

yes I'm under a loop in order to automate datasets moving/renaing; if I use "h5mv" for a single dataset (whatever it is) that's ok. As said previously, the issue occurs immediately after using the first "h5mv" without closing the file.

I'm under Linux (Ubuntu); I've ever tested h5copy but it failed as well.

I'll test your suggestion in using hdf5 package outside Scilab or through "unix_g" ino Scilab

Paul



Le dim. 24 mai 2020 à 14:38, Antoine Monmayrant <[hidden email]> a écrit :

Hello Paul,


This is not a solution, but just tests to assert that the problem does come from scilab and not from your datasets.

(1) Can you try to use command-line tool h5copy (that comes with the hdf5-tools pacakge on a Debian/Ubuntu Linux distribution) to do it outside of scilab?
If you succeed, that might be a bug in h5mv in scilab.
If you are under windows, there might be an equivalent tool, or you can use WSL if under Windows10.

(2) I get that you are running this code inside some sort of loop, right? Can you try to do it step-by-step on one of the file that fails? By step-by-step I mean either Ctrl+E line by line in your code or one command at a time in the console. This might help if it's a race condition bug. I faced similar issues when manipulating the graphic handle hierarchy within a loop.

Hope it helps,


Antoine

On 24/05/2020 12:22, paul francedixhuit wrote:
Hi all

I'm trying to reorganize my hdf5 file by moving/renaming dozens of datasets, using mainly h5mv (loops are used to pass through all  groups and datasets)

My code is typically, where:
  • Source = the path+name of the original dataset
  • Destination= the path + the new dataset name
h5flush(h5);  
h5mv(h5, Source, h5, Destination);

Trouble I'm confront to:
  • It works only ounce and I do not understand why, even flushing hdf5 object
  • I can note that after h5mv, h5flush does not work anymore
  • hdf5 object becomes "invalide"
What am I doing wrong?

Thanks for any advice

Paul

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Antoine Monmayrant Antoine Monmayrant
Reply | Threaded
Open this post in threaded view
|

Re: h5mv / h5flush


On 24/05/2020 15:28, paul francedixhuit wrote:
Thanks Antoine for answering

The datasets work fine.

yes I'm under a loop in order to automate datasets moving/renaing; if I use "h5mv" for a single dataset (whatever it is) that's ok. As said previously, the issue occurs immediately after using the first "h5mv" without closing the file.
OK. I am not sure I understand the issue than.
Is the first call to h5mv OK, then the second one failling?
If it is, did you try h5copen/h5mv/h5close before the next call to h5mv ?

Do you have a minimum working example with an hdf5 file you could share with us?


I'm under Linux (Ubuntu); I've ever tested h5copy but it failed as well.

I'll test your suggestion in using hdf5 package outside Scilab or through "unix_g" ino Scilab
There is also the python api for hdf5 that is really nice and not really difficult to use if you are already familiar with the scilab one.
You could try from python to assess whether you are trying to do something inherently wrong or whether it's a bug in scilab.

Paul



Le dim. 24 mai 2020 à 14:38, Antoine Monmayrant <[hidden email]> a écrit :

Hello Paul,


This is not a solution, but just tests to assert that the problem does come from scilab and not from your datasets.

(1) Can you try to use command-line tool h5copy (that comes with the hdf5-tools pacakge on a Debian/Ubuntu Linux distribution) to do it outside of scilab?
If you succeed, that might be a bug in h5mv in scilab.
If you are under windows, there might be an equivalent tool, or you can use WSL if under Windows10.

(2) I get that you are running this code inside some sort of loop, right? Can you try to do it step-by-step on one of the file that fails? By step-by-step I mean either Ctrl+E line by line in your code or one command at a time in the console. This might help if it's a race condition bug. I faced similar issues when manipulating the graphic handle hierarchy within a loop.

Hope it helps,


Antoine

On 24/05/2020 12:22, paul francedixhuit wrote:
Hi all

I'm trying to reorganize my hdf5 file by moving/renaming dozens of datasets, using mainly h5mv (loops are used to pass through all  groups and datasets)

My code is typically, where:
  • Source = the path+name of the original dataset
  • Destination= the path + the new dataset name
h5flush(h5);  
h5mv(h5, Source, h5, Destination);

Trouble I'm confront to:
  • It works only ounce and I do not understand why, even flushing hdf5 object
  • I can note that after h5mv, h5flush does not work anymore
  • hdf5 object becomes "invalide"
What am I doing wrong?

Thanks for any advice

Paul

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users