[Scilab-users] interp and memory

classic Classic list List threaded Threaded
6 messages Options
jbaudais jbaudais
Reply | Threaded
Open this post in threaded view
|

[Scilab-users] interp and memory

Hello,

   Is my code wrong or is there a real memory problem with the function
interp in the following code? (Scilab fulfills the memory and of course
stops.)

for i=1:1000
   mprintf("%d\n",i);
   n=1e6;
   xp=1:n;
   x=1:100:n;
   y=rand(x);
   d=splin(x,y);
   z=interp(xp,x,y,d);
end

Thanks,

Jean-Yves
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
fmiyara fmiyara
Reply | Threaded
Open this post in threaded view
|

Re: interp and memory


Jean-Ives,

It seems that the variables are not so huge, xp has 1000000 components as well as z, while x and y have 10000 components.

But I guess splin() gets the derivatives through solving a linear equation system of, in this case, 10000 x 10000, and even if the system's matrix is quasi diagonal (it has only the diagonal, sub-diagonal and supra-diagonal components different from 0) and this surely reduces the computational load, it seems quite a large system and probably it gets stuck here.

Probably a lighter (and better) way to do what you seem to be looking for is to try to resample your x-y data by a factor of 100 using intdec(). However, doing that 1000 times may take quite a long time (I haven't tested it). If you really need that, it would
probably be better to do the oversampling algorithm from scratch in such a way to design only once the smoothing filter instead of letting intdec() design the same filter over and over again.

Regards,

Federico Miyara


On 08/03/2021 04:33, Jean-Yves Baudais wrote:
Hello,

  Is my code wrong or is there a real memory problem with the function interp in the following code? (Scilab fulfills the memory and of course stops.)

for i=1:1000
  mprintf("%d\n",i);
  n=1e6;
  xp=1:n;
  x=1:100:n;
  y=rand(x);
  d=splin(x,y);
  z=interp(xp,x,y,d);
end

Thanks,

Jean-Yves
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users



Libre de virus. www.avast.com

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
jbaudais jbaudais
Reply | Threaded
Open this post in threaded view
|

Re: interp and memory

Hi,


> But I guess splin() gets the derivatives [...]

Oh, maybe I was not clear. The memory issue is not here. There is not
problem to compute z. The loop is used to show the problem: it seems
that interp doesn't free the used memory after each call. So after
hundreds of call Scilab is killed!

for i=1:1000
   mprintf("%d\n",i);
   n=1e6;
   xp=1:n;
   x=1:100:n;
   y=rand(x);
   d=splin(x,y);
   z=interp(xp,x,y,d);
end

--Jean-Yves
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Antoine Monmayrant Antoine Monmayrant
Reply | Threaded
Open this post in threaded view
|

Re: interp and memory

Hello Jean-Yves,

Yes, you are right, it does look like a memory leak: the memory
increases linearly with the number of iterations (see attached file and
modified script below).
But keep in mind that the plot might integrate some growing overhead due
to the plotting...

Antoine

/////////

niter=1000;
mems=zeros(1:niter+1)*%nan;
is=0:niter;

h=scf();
mem=evstr(tokens(unix_g('free -b| grep ''Mem:'''))(3));
mems(1)=mem;
plot(is,mems,'k.');
xlabel('iteration of splin/interp')
ylabel('memory used')
for i=1:niter
   mprintf("%d\n",i);
   n=1e6;
   xp=1:n;
   x=1:100:n;
   y=rand(x);
   d=splin(x,y);
   z=interp(xp,x,y,d);
   mem=evstr(tokens(unix_g('free -b| grep ''Mem:'''))(3));
   mems(i+1)=mem;
   plot(is,mems,'k.');
   xs2png(h,"memory_leak.png");
end


Le 08/03/2021 à 09:56, Jean-Yves Baudais a écrit :
> for i=1:1000
>   mprintf("%d\n",i);
>   n=1e6;
>   xp=1:n;
>   x=1:100:n;
>   y=rand(x);
>   d=splin(x,y);
>   z=interp(xp,x,y,d);
> end

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users

memory_leak.png (12K) Download Attachment
Antoine Monmayrant Antoine Monmayrant
Reply | Threaded
Open this post in threaded view
|

Re: interp and memory

In reply to this post by jbaudais
Hello Jean-Yves,

There is a memory leak, in interp, not splin.
Could you create a bug report?

Antoine

///////

test=%t; // no leak if true, leak if false

niter=100;
mems=zeros(1:niter+1)*%nan;
is=0:niter;

h=scf();
mem=evstr(tokens(unix_g('free -b| grep ''Mem:'''))(3));
mems(1)=mem;
plot(is,mems,'k.');
xlabel('iteration of splin/interp')
ylabel('memory used')
for i=1:niter
   mprintf("%d\n",i);
   n=1e6;
   xp=1:n;
   x=1:100:n;
   y=rand(x);
   if test then
       d=splin(x,y);
       z=rand(xp);
   else
      d=splin(x,y);
      z=interp(xp,x,y,d);
   end
   mem=evstr(tokens(unix_g('free -b| grep ''Mem:'''))(3));
   mems(i+1)=mem;
   plot(is,mems,'k.');
   if test then
       xs2png(h,"memory_leak_test.png");
   else
       xs2png(h,"memory_leak.png");
   end

end


Le 08/03/2021 à 09:56, Jean-Yves Baudais a écrit :

> Hi,
>
>
>> But I guess splin() gets the derivatives [...]
>
> Oh, maybe I was not clear. The memory issue is not here. There is not
> problem to compute z. The loop is used to show the problem: it seems
> that interp doesn't free the used memory after each call. So after
> hundreds of call Scilab is killed!
>
> for i=1:1000
>   mprintf("%d\n",i);
>   n=1e6;
>   xp=1:n;
>   x=1:100:n;
>   y=rand(x);
>   d=splin(x,y);
>   z=interp(xp,x,y,d);
> end
>
> --Jean-Yves
> _______________________________________________
> users mailing list
> [hidden email]
> http://lists.scilab.org/mailman/listinfo/users
>
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
mottelet mottelet
Reply | Threaded
Open this post in threaded view
|

Re: interp and memory

It has been fixed by Antoine some minutes ago:

https://codereview.scilab.org/#/c/21708/

S.

Le 08/03/2021 à 10:42, Antoine Monmayrant a écrit :

> Hello Jean-Yves,
>
> There is a memory leak, in interp, not splin.
> Could you create a bug report?
>
> Antoine
>
> ///////
>
> test=%t; // no leak if true, leak if false
>
> niter=100;
> mems=zeros(1:niter+1)*%nan;
> is=0:niter;
>
> h=scf();
> mem=evstr(tokens(unix_g('free -b| grep ''Mem:'''))(3));
> mems(1)=mem;
> plot(is,mems,'k.');
> xlabel('iteration of splin/interp')
> ylabel('memory used')
> for i=1:niter
>   mprintf("%d\n",i);
>   n=1e6;
>   xp=1:n;
>   x=1:100:n;
>   y=rand(x);
>   if test then
>       d=splin(x,y);
>       z=rand(xp);
>   else
>      d=splin(x,y);
>      z=interp(xp,x,y,d);
>   end
>   mem=evstr(tokens(unix_g('free -b| grep ''Mem:'''))(3));
>   mems(i+1)=mem;
>   plot(is,mems,'k.');
>   if test then
>       xs2png(h,"memory_leak_test.png");
>   else
>       xs2png(h,"memory_leak.png");
>   end
>
> end
>
>
> Le 08/03/2021 à 09:56, Jean-Yves Baudais a écrit :
>> Hi,
>>
>>
>>> But I guess splin() gets the derivatives [...]
>>
>> Oh, maybe I was not clear. The memory issue is not here. There is not
>> problem to compute z. The loop is used to show the problem: it seems
>> that interp doesn't free the used memory after each call. So after
>> hundreds of call Scilab is killed!
>>
>> for i=1:1000
>>   mprintf("%d\n",i);
>>   n=1e6;
>>   xp=1:n;
>>   x=1:100:n;
>>   y=rand(x);
>>   d=splin(x,y);
>>   z=interp(xp,x,y,d);
>> end
>>
>> --Jean-Yves
>> _______________________________________________
>> users mailing list
>> [hidden email]
>> https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users 
>>
>>
> _______________________________________________
> users mailing list
> [hidden email]
> https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users 
>

--
Stéphane Mottelet
Ingénieur de recherche
EA 4297 Transformations Intégrées de la Matière Renouvelable
Département Génie des Procédés Industriels
Sorbonne Universités - Université de Technologie de Compiègne
CS 60319, 60203 Compiègne cedex
Tel : +33(0)344234688
http://www.utc.fr/~mottelet

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users