[Scilab-users] scilab 6.1 crashes when trying to load big matfile (loadmatfile)

classic Classic list List threaded Threaded
13 messages Options
Antoine Monmayrant Antoine Monmayrant
Reply | Threaded
Open this post in threaded view
|

[Scilab-users] scilab 6.1 crashes when trying to load big matfile (loadmatfile)

Hello All,

I cannot open large matfile in scilab (~3.4Gb).
Scilab is always dying with an error message that is extremely
instructive: "Killed".

It's a bit cumbersome to share this big fie, so do you have any idea on
how to investigate this issue and try to locate the root cause?

As a side note, it might be a problem related to either the size of the
variables or their nature: cells containing hypermatrices.
The file itself is not corrupted (I managed to open it in matlab and to
save each hypermatrix into individual files that got imported in scilab
with no problem.

Cheers,

Antoine

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
mottelet mottelet
Reply | Threaded
Open this post in threaded view
|

Re: scilab 6.1 crashes when trying to load big matfile (loadmatfile)

Hello Antoine,

Did you to save a single "hypermatrix in a cell" in matlab and then load
in Scilab ?

S.

Le 17/06/2020 à 13:22, Antoine Monmayrant a écrit :

> Hello All,
>
> I cannot open large matfile in scilab (~3.4Gb).
> Scilab is always dying with an error message that is extremely
> instructive: "Killed".
>
> It's a bit cumbersome to share this big fie, so do you have any idea
> on how to investigate this issue and try to locate the root cause?
>
> As a side note, it might be a problem related to either the size of
> the variables or their nature: cells containing hypermatrices.
> The file itself is not corrupted (I managed to open it in matlab and
> to save each hypermatrix into individual files that got imported in
> scilab with no problem.
>
> Cheers,
>
> Antoine
>
> _______________________________________________
> users mailing list
> [hidden email]
> https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users 
>

--
Stéphane Mottelet
Ingénieur de recherche
EA 4297 Transformations Intégrées de la Matière Renouvelable
Département Génie des Procédés Industriels
Sorbonne Universités - Université de Technologie de Compiègne
CS 60319, 60203 Compiègne cedex
Tel : +33(0)344234688
http://www.utc.fr/~mottelet

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Clément David-3 Clément David-3
Reply | Threaded
Open this post in threaded view
|

Re: scilab 6.1 crashes when trying to load big matfile (loadmatfile)

Hello Antoine,

There might be some extra copies at loading time that overflow the memory ; what's the amount of memory you have available ?

And also, to investigate, could you please either share a script used to generate similar data or describe the used data structure ?

Thanks,

Clément

> -----Original Message-----
> From: users <[hidden email]> On Behalf Of Stéphane Mottelet
> Sent: Wednesday, June 17, 2020 1:52 PM
> To: [hidden email]
> Subject: Re: [Scilab-users] scilab 6.1 crashes when trying to load big matfile
> (loadmatfile)
>
> Hello Antoine,
>
> Did you to save a single "hypermatrix in a cell" in matlab and then load in Scilab ?
>
> S.
>
> Le 17/06/2020 à 13:22, Antoine Monmayrant a écrit :
> > Hello All,
> >
> > I cannot open large matfile in scilab (~3.4Gb).
> > Scilab is always dying with an error message that is extremely
> > instructive: "Killed".
> >
> > It's a bit cumbersome to share this big fie, so do you have any idea
> > on how to investigate this issue and try to locate the root cause?
> >
> > As a side note, it might be a problem related to either the size of
> > the variables or their nature: cells containing hypermatrices.
> > The file itself is not corrupted (I managed to open it in matlab and
> > to save each hypermatrix into individual files that got imported in
> > scilab with no problem.
> >
> > Cheers,
> >
> > Antoine
> >
> > _______________________________________________
> > users mailing list
> > [hidden email]
> > https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists
> > .scilab.org/mailman/listinfo/users
> >
>
> --
> Stéphane Mottelet
> Ingénieur de recherche
> EA 4297 Transformations Intégrées de la Matière Renouvelable Département
> Génie des Procédés Industriels Sorbonne Universités - Université de Technologie
> de Compiègne CS 60319, 60203 Compiègne cedex Tel : +33(0)344234688
> http://www.utc.fr/~mottelet
>
> _______________________________________________
> users mailing list
> [hidden email]
> http://lists.scilab.org/mailman/listinfo/users
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Antoine Monmayrant Antoine Monmayrant
Reply | Threaded
Open this post in threaded view
|

Re: scilab 6.1 crashes when trying to load big matfile (loadmatfile)

In reply to this post by mottelet

On 17/06/2020 13:51, Stéphane Mottelet wrote:
> Hello Antoine,
>
> Did you to save a single "hypermatrix in a cell" in matlab and then
> load in Scilab ?
No, just the hypermatrix, not a cell containing only one hypermatrix.

>
> S.
>
> Le 17/06/2020 à 13:22, Antoine Monmayrant a écrit :
>> Hello All,
>>
>> I cannot open large matfile in scilab (~3.4Gb).
>> Scilab is always dying with an error message that is extremely
>> instructive: "Killed".
>>
>> It's a bit cumbersome to share this big fie, so do you have any idea
>> on how to investigate this issue and try to locate the root cause?
>>
>> As a side note, it might be a problem related to either the size of
>> the variables or their nature: cells containing hypermatrices.
>> The file itself is not corrupted (I managed to open it in matlab and
>> to save each hypermatrix into individual files that got imported in
>> scilab with no problem.
>>
>> Cheers,
>>
>> Antoine
>>
>> _______________________________________________
>> users mailing list
>> [hidden email]
>> https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users 
>>
>
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Dang Ngoc Chan, Christophe Dang Ngoc Chan, Christophe
Reply | Threaded
Open this post in threaded view
|

Re: {EXT} scilab 6.1 crashes when trying to load big matfile (loadmatfile)

In reply to this post by Antoine Monmayrant
Hello,

> De la part de Antoine Monmayrant
> Envoyé : mercredi 17 juin 2020 13:22
>
> It's a bit cumbersome to share this big fie, so do you have any idea on how to
> investigate this issue and try to locate the root cause?

You might try to build  a minimal and working (crashing) example,
generating a big file in Scilab (a big ones() or rand() hypermatrix)
and then opening this huge file,
and then share the code (which would be rather small) with us.

Regards

--
Christophe Dang Ngoc Chan
Mechanical calculation engineer

General
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error), please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
mottelet mottelet
Reply | Threaded
Open this post in threaded view
|

Re: scilab 6.1 crashes when trying to load big matfile (loadmatfile)

In reply to this post by Antoine Monmayrant
Hello Antoine,

I made a 4Gb file with Matlab:

 >> a=rand(645,645,645);
 >> b=rand(645,645,645);
 >> c={a,b};
 >> save("c.mat","-7.3","c")

and managed to load it successfully in Scilab:

--> loadmatfile("c.mat");

--> c
  c  =

   [645x645x645 constant]  [645x645x645 constant]

Maybe the structure I tried is too simple (just two hypermatrices in a
cell) so please give us a representative failing example.

S.

Le 17/06/2020 à 13:22, Antoine Monmayrant a écrit :

> Hello All,
>
> I cannot open large matfile in scilab (~3.4Gb).
> Scilab is always dying with an error message that is extremely
> instructive: "Killed".
>
> It's a bit cumbersome to share this big fie, so do you have any idea
> on how to investigate this issue and try to locate the root cause?
>
> As a side note, it might be a problem related to either the size of
> the variables or their nature: cells containing hypermatrices.
> The file itself is not corrupted (I managed to open it in matlab and
> to save each hypermatrix into individual files that got imported in
> scilab with no problem.
>
> Cheers,
>
> Antoine
>
> _______________________________________________
> users mailing list
> [hidden email]
> https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users 
>

--
Stéphane Mottelet
Ingénieur de recherche
EA 4297 Transformations Intégrées de la Matière Renouvelable
Département Génie des Procédés Industriels
Sorbonne Universités - Université de Technologie de Compiègne
CS 60319, 60203 Compiègne cedex
Tel : +33(0)344234688
http://www.utc.fr/~mottelet

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
mottelet mottelet
Reply | Threaded
Open this post in threaded view
|

Re: scilab 6.1 crashes when trying to load big matfile (loadmatfile)

I tried to generate such kind of data file with the following Matlab script:

c1=fillcell();
c2=fillcell();
c3=fillcell();

save("c.mat","-v7.3","c1","c2","c3")

function c = fillcell()
     c={};
     for j=1:11
         c{j}=rand(711,711,33);
     end
end

The c.mat file is 4.18GB and I managed to load it in Scilab.

How much physical memory do you have on your machine ?

S.


Le 18/06/2020 à 13:28, Antoine Monmayrant a écrit :

> Hello,
>
>
> I tried but failed to generate similar data.
> What I had was a mat file containing mainly 3 cells.
> Each Cell contains 11 hypermatrices that are 711x711x31 or 711x711x33.
> I won't have access to the machine that can play with this data in the
> coming days, so I am not sure I can help in the short term.
>
> Antoine
>
> On 18/06/2020 13:06, Stéphane Mottelet wrote:
>> Hello Antoine,
>>
>> I made a 4Gb file with Matlab:
>>
>> >> a=rand(645,645,645);
>> >> b=rand(645,645,645);
>> >> c={a,b};
>> >> save("c.mat","-7.3","c")
>>
>> and managed to load it successfully in Scilab:
>>
>> --> loadmatfile("c.mat");
>>
>> --> c
>>  c  =
>>
>>   [645x645x645 constant]  [645x645x645 constant]
>>
>> Maybe the structure I tried is too simple (just two hypermatrices in
>> a cell) so please give us a representative failing example.
>>
>> S.
>>
>> Le 17/06/2020 à 13:22, Antoine Monmayrant a écrit :
>>> Hello All,
>>>
>>> I cannot open large matfile in scilab (~3.4Gb).
>>> Scilab is always dying with an error message that is extremely
>>> instructive: "Killed".
>>>
>>> It's a bit cumbersome to share this big fie, so do you have any idea
>>> on how to investigate this issue and try to locate the root cause?
>>>
>>> As a side note, it might be a problem related to either the size of
>>> the variables or their nature: cells containing hypermatrices.
>>> The file itself is not corrupted (I managed to open it in matlab and
>>> to save each hypermatrix into individual files that got imported in
>>> scilab with no problem.
>>>
>>> Cheers,
>>>
>>> Antoine
>>>
>>> _______________________________________________
>>> users mailing list
>>> [hidden email]
>>> https://antispam.utc.fr/proxy/2/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users 
>>>
>>
--
Stéphane Mottelet
Ingénieur de recherche
EA 4297 Transformations Intégrées de la Matière Renouvelable
Département Génie des Procédés Industriels
Sorbonne Universités - Université de Technologie de Compiègne
CS 60319, 60203 Compiègne cedex
Tel : +33(0)344234688
http://www.utc.fr/~mottelet

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Antoine Monmayrant Antoine Monmayrant
Reply | Threaded
Open this post in threaded view
|

Re: scilab 6.1 crashes when trying to load big matfile (loadmatfile)

In reply to this post by mottelet
Hello all,


Here is a small script that systematically crashes scilab on my machine:


////////////////////////////////

// on my machine with 8Gb or ram and usual workload, n=28 crashes scilab
n=[24,26,28];

for i=n

     disp('---- '+string(i)+' ----');
     execstr('a'+string(i)+'=[1:2^'+string(i)+'];')
     disp('a'+string(i)+'=[1:2^'+string(i)+'];')
execstr("savematfile(''a"+string(i)+".mat'',''a"+string(i)+"'');")
disp("savematfile(''a"+string(i)+".mat'',''a"+string(i)+"'');")
     execstr("loadmatfile(''a"+string(i)+".mat'');")
     disp("loadmatfile(''a"+string(i)+".mat'');")
     disp('---- OK ----');

end

////////////////////////////////

You'll have to adapt the maximum value for n depending on your available
ram.

Antoine


On 18/06/2020 13:06, Stéphane Mottelet wrote:

> Hello Antoine,
>
> I made a 4Gb file with Matlab:
>
> >> a=rand(645,645,645);
> >> b=rand(645,645,645);
> >> c={a,b};
> >> save("c.mat","-7.3","c")
>
> and managed to load it successfully in Scilab:
>
> --> loadmatfile("c.mat");
>
> --> c
>  c  =
>
>   [645x645x645 constant]  [645x645x645 constant]
>
> Maybe the structure I tried is too simple (just two hypermatrices in a
> cell) so please give us a representative failing example.
>
> S.
>
> Le 17/06/2020 à 13:22, Antoine Monmayrant a écrit :
>> Hello All,
>>
>> I cannot open large matfile in scilab (~3.4Gb).
>> Scilab is always dying with an error message that is extremely
>> instructive: "Killed".
>>
>> It's a bit cumbersome to share this big fie, so do you have any idea
>> on how to investigate this issue and try to locate the root cause?
>>
>> As a side note, it might be a problem related to either the size of
>> the variables or their nature: cells containing hypermatrices.
>> The file itself is not corrupted (I managed to open it in matlab and
>> to save each hypermatrix into individual files that got imported in
>> scilab with no problem.
>>
>> Cheers,
>>
>> Antoine
>>
>> _______________________________________________
>> users mailing list
>> [hidden email]
>> https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users 
>>
>
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
mottelet mottelet
Reply | Threaded
Open this post in threaded view
|

Re: scilab 6.1 crashes when trying to load big matfile (loadmatfile)

it's the loadmatfile that crashes (here on Scilab-branch-6.1 under OSX,
with 16Gb ram)

   "savematfile('a28.mat','a28');"
scilab-cli-bin(3310,0x115f1f5c0) malloc: can't allocate region
*** mach_vm_map(size=18446744071562067968) failed (error code=3)
scilab-cli-bin(3310,0x115f1f5c0) malloc: *** set a breakpoint in
malloc_error_break to debug

Same crash with Scilab-branch-6.1 under Ubuntu 18.04, with 128Gb ram:

   "a28=[1:2^28];"
ATTENTION : Option -v7 ajoutée.

   "savematfile('a28.mat','a28');"
Segmentation fault (core dumped)

S.

Le 19/06/2020 à 08:57, Antoine Monmayrant a écrit :

> Hello all,
>
>
> Here is a small script that systematically crashes scilab on my machine:
>
>
> ////////////////////////////////
>
> // on my machine with 8Gb or ram and usual workload, n=28 crashes scilab
> n=[24,26,28];
>
> for i=n
>
>     disp('---- '+string(i)+' ----');
>     execstr('a'+string(i)+'=[1:2^'+string(i)+'];')
>     disp('a'+string(i)+'=[1:2^'+string(i)+'];')
> execstr("savematfile(''a"+string(i)+".mat'',''a"+string(i)+"'');")
> disp("savematfile(''a"+string(i)+".mat'',''a"+string(i)+"'');")
>     execstr("loadmatfile(''a"+string(i)+".mat'');")
>     disp("loadmatfile(''a"+string(i)+".mat'');")
>     disp('---- OK ----');
>
> end
>
> ////////////////////////////////
>
> You'll have to adapt the maximum value for n depending on your
> available ram.
>
> Antoine
>
>
> On 18/06/2020 13:06, Stéphane Mottelet wrote:
>> Hello Antoine,
>>
>> I made a 4Gb file with Matlab:
>>
>> >> a=rand(645,645,645);
>> >> b=rand(645,645,645);
>> >> c={a,b};
>> >> save("c.mat","-7.3","c")
>>
>> and managed to load it successfully in Scilab:
>>
>> --> loadmatfile("c.mat");
>>
>> --> c
>>  c  =
>>
>>   [645x645x645 constant]  [645x645x645 constant]
>>
>> Maybe the structure I tried is too simple (just two hypermatrices in
>> a cell) so please give us a representative failing example.
>>
>> S.
>>
>> Le 17/06/2020 à 13:22, Antoine Monmayrant a écrit :
>>> Hello All,
>>>
>>> I cannot open large matfile in scilab (~3.4Gb).
>>> Scilab is always dying with an error message that is extremely
>>> instructive: "Killed".
>>>
>>> It's a bit cumbersome to share this big fie, so do you have any idea
>>> on how to investigate this issue and try to locate the root cause?
>>>
>>> As a side note, it might be a problem related to either the size of
>>> the variables or their nature: cells containing hypermatrices.
>>> The file itself is not corrupted (I managed to open it in matlab and
>>> to save each hypermatrix into individual files that got imported in
>>> scilab with no problem.
>>>
>>> Cheers,
>>>
>>> Antoine
>>>
>>> _______________________________________________
>>> users mailing list
>>> [hidden email]
>>> https://antispam.utc.fr/proxy/2/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users 
>>>
>>
> _______________________________________________
> users mailing list
> [hidden email]
> https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users 
>

--
Stéphane Mottelet
Ingénieur de recherche
EA 4297 Transformations Intégrées de la Matière Renouvelable
Département Génie des Procédés Industriels
Sorbonne Universités - Université de Technologie de Compiègne
CS 60319, 60203 Compiègne cedex
Tel : +33(0)344234688
http://www.utc.fr/~mottelet

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Dang Ngoc Chan, Christophe Dang Ngoc Chan, Christophe
Reply | Threaded
Open this post in threaded view
|

Re: {EXT} Re: scilab 6.1 crashes when trying to load big matfile (loadmatfile)

In reply to this post by Antoine Monmayrant
Hello,

> De : De la part de Antoine Monmayrant
> Envoyé : vendredi 19 juin 2020 08:58
>
> Here is a small script that systematically crashes scilab on my machine:

It works fine for me, no crash.
Windows 7, RAM 16 GB, 12.9 GB used at peak.

[a, b] = getdebuginfo()
 a  =

  "Memory in use:      39 %"
  "Total Physical Memory (Kbytes): 16673068"
  "Free Physical Memory (Kbytes): 10007928"
  "Total Paging File (Kbytes): 33344236"
  "Free Paging File (Kbytes): 24405280"
  "Total Virtual Memory (Kbytes): 8589934464"
  "Free Virtual Memory (Kbytes): 8586180260"
  "Free Extended Memory (Kbytes):       0"
  "Operating System: Windows Seven x64"
  "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz"
  "Number of processors: 8"
  "Number of Video cards: 1"
  "Video card #0: NVIDIA Quadro K2200"
  "Primary Video card driver version: 24.21.14.1163"
  "Screen size: 1920 x 1200 32 bits"
  "Number of Monitors: 2"
  "Path: [...]
 b  =

  "Version: scilab-6.1.0"
  "Compilation date: Feb 25 2020"
  "Compilation time: 11:34:32"
  "Compiler Architecture: X64"
  "Compiled with Microsoft compiler (191627035)"
  "BLAS library optimized version: MKL"
  "XML version: 2.9.1"
  "Tcl/Tk: Enable"
  "TCL version: 8.5.9"
  "TK version: 8.5.9"
  "Path separator: ;"
  "Directory separator: \"
  "PCRE Version: 8.21"

--
Christophe Dang Ngoc Chan
Mechanical calculation engineer

General
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error), please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Rafael Guerra Rafael Guerra
Reply | Threaded
Open this post in threaded view
|

Re: scilab 6.1 crashes when trying to load big matfile (loadmatfile)

In reply to this post by mottelet
Script runs fine on Win10, Scilab 6.1, 64 GB RAM.
But btw, the file a28.mat seems to have only 342MB.


-----Original Message-----
From: users <[hidden email]> On Behalf Of Stéphane Mottelet
Sent: Friday, June 19, 2020 10:15 AM
To: [hidden email]
Subject: Re: [Scilab-users] scilab 6.1 crashes when trying to load big matfile (loadmatfile)

it's the loadmatfile that crashes (here on Scilab-branch-6.1 under OSX,
with 16Gb ram)

   "savematfile('a28.mat','a28');"
scilab-cli-bin(3310,0x115f1f5c0) malloc: can't allocate region
*** mach_vm_map(size=18446744071562067968) failed (error code=3)
scilab-cli-bin(3310,0x115f1f5c0) malloc: *** set a breakpoint in
malloc_error_break to debug

Same crash with Scilab-branch-6.1 under Ubuntu 18.04, with 128Gb ram:

   "a28=[1:2^28];"
ATTENTION : Option -v7 ajoutée.

   "savematfile('a28.mat','a28');"
Segmentation fault (core dumped)

S.

Le 19/06/2020 à 08:57, Antoine Monmayrant a écrit :

> Hello all,
>
>
> Here is a small script that systematically crashes scilab on my machine:
>
>
> ////////////////////////////////
>
> // on my machine with 8Gb or ram and usual workload, n=28 crashes scilab
> n=[24,26,28];
>
> for i=n
>
>     disp('---- '+string(i)+' ----');
>     execstr('a'+string(i)+'=[1:2^'+string(i)+'];')
>     disp('a'+string(i)+'=[1:2^'+string(i)+'];')
> execstr("savematfile(''a"+string(i)+".mat'',''a"+string(i)+"'');")
> disp("savematfile(''a"+string(i)+".mat'',''a"+string(i)+"'');")
>     execstr("loadmatfile(''a"+string(i)+".mat'');")
>     disp("loadmatfile(''a"+string(i)+".mat'');")
>     disp('---- OK ----');
>
> end
>

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Antoine Monmayrant-2 Antoine Monmayrant-2
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? scilab 6.1 crashes when trying to load big matfile (loadmatfile)

Le Vendredi, Juin 19, 2020 15:59 CEST, Rafael Guerra <[hidden email]> a écrit:
 
> Script runs fine on Win10, Scilab 6.1, 64 GB RAM.
> But btw, the file a28.mat seems to have only 342MB.

I suspect that it does work fine with 64GB of ram: that's how I managed to work around my initial problem.
This script should be adapted to your available ram: you should try to increase n.
But in your case, I think you have enough ram to reach scilab maximum variable size (is it something like 2^31-1 maximum elements?) before triggering this bug.
Ideally, this should be tested on a ram-limited hardware (~8GB).
The issue is that there might be some malloc() somewhere that fails, while the code tries to use the memory without checking whether the malloc() was successful or not.
If you have gazillion's of GB of available ram, you never face this problem.

Antoine

>
>
> -----Original Message-----
> From: users <[hidden email]> On Behalf Of Stéphane Mottelet
> Sent: Friday, June 19, 2020 10:15 AM
> To: [hidden email]
> Subject: Re: [Scilab-users] scilab 6.1 crashes when trying to load big matfile (loadmatfile)
>
> it's the loadmatfile that crashes (here on Scilab-branch-6.1 under OSX,
> with 16Gb ram)
>
>    "savematfile('a28.mat','a28');"
> scilab-cli-bin(3310,0x115f1f5c0) malloc: can't allocate region
> *** mach_vm_map(size=18446744071562067968) failed (error code=3)
> scilab-cli-bin(3310,0x115f1f5c0) malloc: *** set a breakpoint in
> malloc_error_break to debug
>
> Same crash with Scilab-branch-6.1 under Ubuntu 18.04, with 128Gb ram:
>
>    "a28=[1:2^28];"
> ATTENTION : Option -v7 ajoutée.
>
>    "savematfile('a28.mat','a28');"
> Segmentation fault (core dumped)
>
> S.
>
> Le 19/06/2020 à 08:57, Antoine Monmayrant a écrit :
> > Hello all,
> >
> >
> > Here is a small script that systematically crashes scilab on my machine:
> >
> >
> > ////////////////////////////////
> >
> > // on my machine with 8Gb or ram and usual workload, n=28 crashes scilab
> > n=[24,26,28];
> >
> > for i=n
> >
> >     disp('---- '+string(i)+' ----');
> >     execstr('a'+string(i)+'=[1:2^'+string(i)+'];')
> >     disp('a'+string(i)+'=[1:2^'+string(i)+'];')
> > execstr("savematfile(''a"+string(i)+".mat'',''a"+string(i)+"'');")
> > disp("savematfile(''a"+string(i)+".mat'',''a"+string(i)+"'');")
> >     execstr("loadmatfile(''a"+string(i)+".mat'');")
> >     disp("loadmatfile(''a"+string(i)+".mat'');")
> >     disp('---- OK ----');
> >
> > end
> >
>
> _______________________________________________
> users mailing list
> [hidden email]
> http://lists.scilab.org/mailman/listinfo/users
>

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
mottelet mottelet
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? scilab 6.1 crashes when trying to load big matfile (loadmatfile)



> Le 20 juin 2020 à 03:00, Antoine Monmayrant <[hidden email]> a écrit :
>
> Le Vendredi, Juin 19, 2020 15:59 CEST, Rafael Guerra <[hidden email]> a écrit:
>
>> Script runs fine on Win10, Scilab 6.1, 64 GB RAM.
>> But btw, the file a28.mat seems to have only 342MB.
>
> I suspect that it does work fine with 64GB of ram:

Not so sure. It crashes on a 128GB machine.

S.


> that's how I managed to work around my initial problem.
> This script should be adapted to your available ram: you should try to increase n.
> But in your case, I think you have enough ram to reach scilab maximum variable size (is it something like 2^31-1 maximum elements?) before triggering this bug.
> Ideally, this should be tested on a ram-limited hardware (~8GB).
> The issue is that there might be some malloc() somewhere that fails, while the code tries to use the memory without checking whether the malloc() was successful or not.
> If you have gazillion's of GB of available ram, you never face this problem.
>
> Antoine
>
>>
>>
>> -----Original Message-----
>> From: users <[hidden email]> On Behalf Of Stéphane Mottelet
>> Sent: Friday, June 19, 2020 10:15 AM
>> To: [hidden email]
>> Subject: Re: [Scilab-users] scilab 6.1 crashes when trying to load big matfile (loadmatfile)
>>
>> it's the loadmatfile that crashes (here on Scilab-branch-6.1 under OSX,
>> with 16Gb ram)
>>
>>   "savematfile('a28.mat','a28');"
>> scilab-cli-bin(3310,0x115f1f5c0) malloc: can't allocate region
>> *** mach_vm_map(size=18446744071562067968) failed (error code=3)
>> scilab-cli-bin(3310,0x115f1f5c0) malloc: *** set a breakpoint in
>> malloc_error_break to debug
>>
>> Same crash with Scilab-branch-6.1 under Ubuntu 18.04, with 128Gb ram:
>>
>>   "a28=[1:2^28];"
>> ATTENTION : Option -v7 ajoutée.
>>
>>   "savematfile('a28.mat','a28');"
>> Segmentation fault (core dumped)
>>
>> S.
>>
>>> Le 19/06/2020 à 08:57, Antoine Monmayrant a écrit :
>>> Hello all,
>>>
>>>
>>> Here is a small script that systematically crashes scilab on my machine:
>>>
>>>
>>> ////////////////////////////////
>>>
>>> // on my machine with 8Gb or ram and usual workload, n=28 crashes scilab
>>> n=[24,26,28];
>>>
>>> for i=n
>>>
>>>     disp('---- '+string(i)+' ----');
>>>     execstr('a'+string(i)+'=[1:2^'+string(i)+'];')
>>>     disp('a'+string(i)+'=[1:2^'+string(i)+'];')
>>> execstr("savematfile(''a"+string(i)+".mat'',''a"+string(i)+"'');")
>>> disp("savematfile(''a"+string(i)+".mat'',''a"+string(i)+"'');")
>>>     execstr("loadmatfile(''a"+string(i)+".mat'');")
>>>     disp("loadmatfile(''a"+string(i)+".mat'');")
>>>     disp('---- OK ----');
>>>
>>> end
>>>
>>
>> _______________________________________________
>> users mailing list
>> [hidden email]
>> https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users
>>
>
> _______________________________________________
> users mailing list
> [hidden email]
> https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users