[Scilab-users] parallel_run

classic Classic list List threaded Threaded
12 messages Options
fujimoto2005 fujimoto2005
Reply | Threaded
Open this post in threaded view
|

[Scilab-users] parallel_run

I wanted to use "parallel_run" and tried the sample script in the help page.
But it did not work with the error message "undefined
variable:parallel_run".
Please teach me what is wrong?

The sample script I used is as follows
******
for i = 1:10
  res(i) = i*i;
end;
// for parallel_run, we need to have a function performing the computation
function a=g(arg1)
  a=arg1*arg1
endfunction

res = parallel_run(1:10, g);
// => res = [1  4  9  16  25  36  49  64  81  100];
*********





--
Sent from: http://mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
amonmayr amonmayr
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? ?==?utf-8?q? parallel_run

Hello,

Are you using Windows or anything else?
Because as far as I remember, parallel_run is just a dirty hack that only works under windows.
On my linux install of scilab 6.0.2, I also have:
 --> parallel_run

Undefined variable: parallel_run

So it might be a bug in the documentation that should mention "windows only" like some other functions.

Antoine
 
 
Le Lundi, Septembre 23, 2019 15:46 CEST, fujimoto2005 <[hidden email]> a écrit:
 

> I wanted to use "parallel_run" and tried the sample script in the help page.
> But it did not work with the error message "undefined
> variable:parallel_run".
> Please teach me what is wrong?
>
> The sample script I used is as follows
> ******
> for i = 1:10
>   res(i) = i*i;
> end;
> // for parallel_run, we need to have a function performing the computation
> function a=g(arg1)
>   a=arg1*arg1
> endfunction
>
> res = parallel_run(1:10, g);
> // => res = [1  4  9  16  25  36  49  64  81  100];
> *********
>
>
>
>
>
> --
> Sent from: http://mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
> _______________________________________________
> users mailing list
> [hidden email]
> http://lists.scilab.org/mailman/listinfo/users
>

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
fujimoto2005 fujimoto2005
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? ?==?utf-8?q? parallel_run

Dear  Antoine

I am using  windows  7.

Best regards



--
Sent from: http://mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
amonmayr amonmayr
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q?

OK, so it might not be os specific.
It might be a good idea to fill a bug report to say that parallel_run is not available, contrary to what's in the documentation.
If you fill a bug, I'll be happy to confirm it for the linux side of the world...

Antoine
 
 
Le Lundi, Septembre 23, 2019 18:03 CEST, fujimoto2005 <[hidden email]> a écrit:
 

> Dear  Antoine
>
> I am using  windows  7.
>
> Best regards
>
>
>
> --
> Sent from: http://mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
> _______________________________________________
> users mailing list
> [hidden email]
> http://lists.scilab.org/mailman/listinfo/users
>

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
RolandB RolandB
Reply | Threaded
Open this post in threaded view
|

Re: Parallel Run

Hi,

Which version of Scilab do you use?

AFAIK parallel run does not work in Scilab 6.x

It only worked on Scilab 5 and there it would also only work under Linux.

Regards
Roland



--
Sent from: http://mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
mottelet mottelet
Reply | Threaded
Open this post in threaded view
|

Re: ?= Re: ?= =?utf-8?q? parallel_run

In reply to this post by amonmayr

Hello Antoine,

Before judging that some code is just a "dirty hack" you should first get into the code and see hows things are written. The implementation in 5.5.2 was using OpenMP so I don't see anything dirty here. The implementation was unstable under OSX and Linux, and under Windows parallel_run was just making calls sequentially. So no "dirty hack" under Windows but just a shortcut. Octave does also the same with Matlatb's parfor : it just does things sequentially. However, even unstable, parallel_run was very useful (I used it a lot under Linux to do some Monte Carlo estimation on a 40 processor machine). It is a pity that it's re-engineering for 6.0 familiy was not a priority. 

S. 

Quoting Antoine Monmayrant <[hidden email]>:

Hello,

Are you using Windows or anything else?
Because as far as I remember, parallel_run is just a dirty hack that only works under windows.
On my linux install of scilab 6.0.2, I also have:
--> parallel_run

Undefined variable: parallel_run

So it might be a bug in the documentation that should mention "windows only" like some other functions.

Antoine


Le Lundi, Septembre 23, 2019 15:46 CEST, fujimoto2005 <[hidden email]> a écrit:

I wanted to use "parallel_run" and tried the sample script in the help page.
But it did not work with the error message "undefined
variable:parallel_run".
Please teach me what is wrong?

The sample script I used is as follows
******
for i = 1:10
  res(i) = i*i;
end;
// for parallel_run, we need to have a function performing the computation
function a=g(arg1)
  a=arg1*arg1
endfunction

res = parallel_run(1:10, g);
// => res = [1  4  9  16  25  36  49  64  81  100];
*********





--
Sent from: https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email].<a href="orghttps://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users" target="_blank">orghttps://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users




_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
mottelet mottelet
Reply | Threaded
Open this post in threaded view
|

Re: ?=Re: ?=Re: ?=Re: ?==?utf-8?q?

In reply to this post by amonmayr

See http://bugzilla.scilab.org/show_bug.cgi?id=14486

Quoting Antoine Monmayrant <[hidden email]>:

OK, so it might not be os specific.
It might be a good idea to fill a bug report to say that parallel_run is not available, contrary to what's in the documentation.
If you fill a bug, I'll be happy to confirm it for the linux side of the world...

Antoine


Le Lundi, Septembre 23, 2019 18:03 CEST, fujimoto2005 <[hidden email]> a écrit:

Dear  Antoine

I am using  windows  7.

Best regards



--
Sent from: https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email].<a href="orghttps://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users" target="_blank">orghttps://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users




_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
fujimoto2005 fujimoto2005
Reply | Threaded
Open this post in threaded view
|

Re: ?=Re: ?=Re: ?=Re: ?==?utf-8?q?

This post was updated on .
Dear all,

Thanks for a lot of information.

If the documentation contained information on the workable version, it would
be much better for users.
Now the almost documentations have a warning message "Please note that the
recommended version of Scilab is 6.0.2. This page might be outdated. See the
recommended documentation of this function".

Such warning message is very helpful for the special function workable for
only the old version.

By the way is there any parallel computing tool for version 6.0.2?
I am solving the stochastic control problem.
The problem has about 20,000 lattice nodes with 2 dimensions for each time
step and time steps are 15.
Though the calculation on one node is independent each other in each time step,
it uses "optim" and "intg" so that the vectorization of the node is
impossible.
I use loop 300 thousand times.
Any tools of parallel computing are welcome.

Best regards,

Masahiro Fujimoto



--
Sent from: http://mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
users@lists.scilab.org
http://lists.scilab.org/mailman/listinfo/users
mottelet mottelet
Reply | Threaded
Open this post in threaded view
|

Re: ?=Re: ?=Re: ?=Re: ?==?utf-8?q?

Hello,

Le 24/09/2019 à 02:48, fujimoto2005 a écrit :
> Dear all,
>
> Thanks for a lot of information.
>
> If the documentation contained information on the workable version, it would
> be much better for users.

Feel free to help making the documentation better. Scilab is free
software and cannot be improved if active users do not start  to help.

> Now the almost documentations have a warning message "Please note that the
> recommended version of Scilab is 6.0.2. This page might be outdated. See the
> recommended documentation of this function".
>
> Such warning message is very helpful for the special function workable for
> only the old version.
>
> By the way is there any parallel computing tool for version 6.0.2?
no
> I am solving the stochastic control problem.
> The problem has about 20,000 lattice nodes with 2 dimensions for each time
> step, and time steps are 15.
> Though calculation on one node is independent each other in each time step,
> it uses "optim" and "intg" so that the vectorization of the node is
> impossible.
> I use loop 300 thousand times.
> Any tools of paralell conputing are welcome.

The only possible/workable setup is : Scilab 5.5.2 + Linux

S.

--
Stéphane Mottelet
Ingénieur de recherche
EA 4297 Transformations Intégrées de la Matière Renouvelable
Département Génie des Procédés Industriels
Sorbonne Universités - Université de Technologie de Compiègne
CS 60319, 60203 Compiègne cedex
Tel : +33(0)344234688
http://www.utc.fr/~mottelet

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Antoine Monmayrant Antoine Monmayrant
Reply | Threaded
Open this post in threaded view
|

Re: ?= Re: ?= =?utf-8?q? parallel_run

In reply to this post by mottelet


Le 23/09/2019 à 23:13, [hidden email] a écrit :

Hello Antoine,

Before judging that some code is just a "dirty hack" you should first get into the code and see hows things are written. The implementation in 5.5.2 was using OpenMP so I don't see anything dirty here. The implementation was unstable under OSX and Linux, and under Windows parallel_run was just making calls sequentially. So no "dirty hack" under Windows but just a shortcut. Octave does also the same with Matlatb's parfor : it just does things sequentially. However, even unstable, parallel_run was very useful (I used it a lot under Linux to do some Monte Carlo estimation on a 40 processor machine). It is a pity that it's re-engineering for 6.0 familiy was not a priority.

Hello Stephane,

I'm sorry, I did not want to sound rude calling it a "dirty hack".
It's just the explanation I got from scilab developers last time I discussed with them about it.
We were trying to get similar performances under linux/osx/windows with some code using parallel_run.
That was a dead end.

Having any sort of parallel computing under scilab 6.0.2 would be a real step forward for us.
I have played a bit with julia to see what it means parallelizing some code in julia.
It was quite easy.
I don't know whether a similar approach can be implemented in scilab.


Antoine


S. 

Quoting Antoine Monmayrant <[hidden email]>:

Hello,

Are you using Windows or anything else?
Because as far as I remember, parallel_run is just a dirty hack that only works under windows.
On my linux install of scilab 6.0.2, I also have:
--> parallel_run

Undefined variable: parallel_run

So it might be a bug in the documentation that should mention "windows only" like some other functions.

Antoine


Le Lundi, Septembre 23, 2019 15:46 CEST, fujimoto2005 <[hidden email]> a écrit:

I wanted to use "parallel_run" and tried the sample script in the help page.
But it did not work with the error message "undefined
variable:parallel_run".
Please teach me what is wrong?

The sample script I used is as follows
******
for i = 1:10
  res(i) = i*i;
end;
// for parallel_run, we need to have a function performing the computation
function a=g(arg1)
  a=arg1*arg1
endfunction

res = parallel_run(1:10, g);
// => res = [1  4  9  16  25  36  49  64  81  100];
*********





--
Sent from: https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email].<a href="orghttps://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users" target="_blank" moz-do-not-send="true">orghttps://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users




_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Arvid Rosén Arvid Rosén
Reply | Threaded
Open this post in threaded view
|

Re: ?= Re: ?= =?utf-8?q? parallel_run

Hi,

 

I agree. The lack of a working solution for parallel computations is a problem on modern server type machines with many cores. We typically try to work around this by having scilab functions that generate scilab scripts that are executed by spawned headless scilab processes. Communication is achieved by writing and reading files.

 

Needless to say, this is not a great solution, and with scilab 6 it is starting to become unpractical as writing and reading hdf files is extremely slow compared to the scilab 5 file format.

 

Cheers,

Arvid

 

 

From: Scilab Users List <[hidden email]> on behalf of Antoine Monmayrant <[hidden email]>
Reply to: Users mailing list for Scilab <[hidden email]>
Date: Tuesday, 24 September 2019 at 09:32
To: "[hidden email]" <[hidden email]>
Subject: Re: [Scilab-users] ?= Re: ?= =?utf-8?q? parallel_run

 

 

Le 23/09/2019 à 23:13, [hidden email] a écrit :

Hello Antoine,

Before judging that some code is just a "dirty hack" you should first get into the code and see hows things are written. The implementation in 5.5.2 was using OpenMP so I don't see anything dirty here. The implementation was unstable under OSX and Linux, and under Windows parallel_run was just making calls sequentially. So no "dirty hack" under Windows but just a shortcut. Octave does also the same with Matlatb's parfor : it just does things sequentially. However, even unstable, parallel_run was very useful (I used it a lot under Linux to do some Monte Carlo estimation on a 40 processor machine). It is a pity that it's re-engineering for 6.0 familiy was not a priority.

Hello Stephane,

I'm sorry, I did not want to sound rude calling it a "dirty hack".
It's just the explanation I got from scilab developers last time I discussed with them about it.
We were trying to get similar performances under linux/osx/windows with some code using parallel_run.
That was a dead end.

Having any sort of parallel computing under scilab 6.0.2 would be a real step forward for us.
I have played a bit with julia to see what it means parallelizing some code in julia.
It was quite easy.
I don't know whether a similar approach can be implemented in scilab.

 

Antoine


S. 

Quoting Antoine Monmayrant <[hidden email]>:

Hello,

Are you using Windows or anything else?
Because as far as I remember, parallel_run is just a dirty hack that only works under windows.
On my linux install of scilab 6.0.2, I also have:
--> parallel_run

Undefined variable: parallel_run

So it might be a bug in the documentation that should mention "windows only" like some other functions.

Antoine


Le Lundi, Septembre 23, 2019 15:46 CEST, fujimoto2005 <[hidden email]> a écrit:

I wanted to use "parallel_run" and tried the sample script in the help page.
But it did not work with the error message "undefined
variable:parallel_run".
Please teach me what is wrong?

The sample script I used is as follows
******
for i = 1:10
  res(i) = i*i;
end;
// for parallel_run, we need to have a function performing the computation
function a=g(arg1)
  a=arg1*arg1
endfunction

res = parallel_run(1:10, g);
// => res = [1  4  9  16  25  36  49  64  81  100];
*********





--
Sent from: https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email].<a href="orghttps://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users" target="_blank">orghttps://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users

 



_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
mottelet mottelet
Reply | Threaded
Open this post in threaded view
|

Re: ?= Re: ?= =?utf-8?q? parallel_run

In reply to this post by Antoine Monmayrant


Le 24/09/2019 à 09:29, Antoine Monmayrant a écrit :


Le 23/09/2019 à 23:13, [hidden email] a écrit :

Hello Antoine,

Before judging that some code is just a "dirty hack" you should first get into the code and see hows things are written. The implementation in 5.5.2 was using OpenMP so I don't see anything dirty here. The implementation was unstable under OSX and Linux, and under Windows parallel_run was just making calls sequentially. So no "dirty hack" under Windows but just a shortcut. Octave does also the same with Matlatb's parfor : it just does things sequentially. However, even unstable, parallel_run was very useful (I used it a lot under Linux to do some Monte Carlo estimation on a 40 processor machine). It is a pity that it's re-engineering for 6.0 familiy was not a priority.

Hello Stephane,

I'm sorry, I did not want to sound rude calling it a "dirty hack".
It's just the explanation I got from scilab developers last time I discussed with them about it.
We were trying to get similar performances under linux/osx/windows with some code using parallel_run.

Under OSX parallel_run used to work (a little bit) up to 5.5.1 version, then it was almost unusable.

That was a dead end.

Having any sort of parallel computing under scilab 6.0.2 would be a real step forward for us.
I have played a bit with julia to see what it means parallelizing some code in julia.
It was quite easy.

Yeah, Julia has many interesting features but things have been developped with such a kind of orthodoxy that using it is sometimes very baffling...

I don't know whether a similar approach can be implemented in scilab.

The rewritten internals (ast module) would allow to implement (at least) something similar to Matlab's parfor. parallel_run worked by forking the main Scilab process, that's why it was not transposable to Windows (no fork()). Maybe could it be possible with threads with the new architecture ? The only thing that's certain is that without an active collaboration of Antoine Elias this feature will never be developped. The only way is to persuade ESI that it is a strategic feature for their business with Scilab !

S.


Antoine


S. 

Quoting Antoine Monmayrant <[hidden email]>:

Hello,

Are you using Windows or anything else?
Because as far as I remember, parallel_run is just a dirty hack that only works under windows.
On my linux install of scilab 6.0.2, I also have:
--> parallel_run

Undefined variable: parallel_run

So it might be a bug in the documentation that should mention "windows only" like some other functions.

Antoine


Le Lundi, Septembre 23, 2019 15:46 CEST, fujimoto2005 <[hidden email]> a écrit:

I wanted to use "parallel_run" and tried the sample script in the help page.
But it did not work with the error message "undefined
variable:parallel_run".
Please teach me what is wrong?

The sample script I used is as follows
******
for i = 1:10
  res(i) = i*i;
end;
// for parallel_run, we need to have a function performing the computation
function a=g(arg1)
  a=arg1*arg1
endfunction

res = parallel_run(1:10, g);
// => res = [1  4  9  16  25  36  49  64  81  100];
*********





--
Sent from: https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email].<a href="orghttps://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users" target="_blank" moz-do-not-send="true">orghttps://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users




_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email]
https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users
-- 
Stéphane Mottelet
Ingénieur de recherche
EA 4297 Transformations Intégrées de la Matière Renouvelable
Département Génie des Procédés Industriels
Sorbonne Universités - Université de Technologie de Compiègne
CS 60319, 60203 Compiègne cedex
Tel : +33(0)344234688
http://www.utc.fr/~mottelet

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users