Tcl and Scilab Event loop

classic Classic list List threaded Threaded
11 messages Options
Bruno JOFRET Bruno JOFRET
Reply | Threaded
Open this post in threaded view
|

Tcl and Scilab Event loop

Hi all puffin friends !

First of all, I wish all of you my best wishes for 2008 : health and
happiness !
And obviously a great success for Scilab 5 and beyond.

I'm now working on the Tcl running in Scilab and the "Scilab Event Loop".
I confess with the new Java GUI, TCL based stuff were muddy.

Some answers on that issue :
- Tcl based GUI needs manually "update" in order to live.
Of course because Tcl is a scripting language.
-  Scilab was strongly single threaded and running in an endless loop
(that's why it was able to update Tcl/Tk periodically)

Now with the Java gui the "update" loop was slowed down (not so often
called).
This explain why it looks so muddy.

The main idea of my work was to unlink scilab internal behaviour and Tcl
management.

For that issue I created the TCL Interpreter used in Scilab in it's
single thread.
It works as follow :
* I created a timer thread within this one
* WHILE Tcl is started
    IF we have a TCL command to execute
       runIt
    ELSE
       Do "update"
       Wait until we receive a "wake up" signal
    ENDIF
ENDWHILE

The timer thread is quite dummy.
It only send  periodically a "wake up" signal.

"Wake up" signal are also send when asking for a TCL command through Scilab.
A call to Tcl_EvalStr for instance.

It works pretty well on my computer but has some restrictions.
- I used pthread that is not supported on Windows.
But we are working to native similar thread management on Windows.
- Some others commands may be broken.

Some improvements are quite cool :
- You can now run some huge stuff in Scilab, and continue to write in
Scipad.
- No more muddy help browser.
- Whole Scilab behaviour is decreasing computation power need.

Keeping in mind I absolutely do not want to break any OS Scilab version,
neither any Scilab Tool (Scipad, ged, ...) ,
- What do you think of that "algorithm" and did you see any hidden
drawbacks within it ?
- As soon as we will have Windows similar functionalities, and taking
into account your comments,
we will include this into the Trunk and kindly ask you for some feedbacks.


Regards,

--
Bruno JOFRET

         Project Engineer
___SCILAB - INRIA Rocquencourt___
  Domaine de Voluceau - B.P. 105
    78153 Le Chesnay Cedex
  Tel : (+33/0)1.39.63.58.63
  Mailto : [hidden email]
     http://www.scilab.org

Francois Vogel-2 Francois Vogel-2
Reply | Threaded
Open this post in threaded view
|

Re: Tcl and Scilab Event loop

Hi all,

Improving the Tcl/Scilab interface in trunk was indeed badly needed.
The main issue is (or was, perhaps?) bug 2514 and 2640, which prevent
from using anything in Tcl within the trunk. So, I wish a warm welcome
to this work!


Bruno JOFRET said on 11/01/2008 11:07:

> It works as follow :
> * I created a timer thread within this one
> * WHILE Tcl is started
>    IF we have a TCL command to execute
>       runIt
>    ELSE
>       Do "update"
>       Wait until we receive a "wake up" signal
>    ENDIF
> ENDWHILE

The above is oriented towards having responsive Tcl interfaces while
Scilab is busy running, and that's the main goal of course.

However, the Tcl command that gets run in the "runIt" line above might
take time to execute. Say that the following command is sent from a
Tcl script:

ScilabEval "j=0;for i=1:1e6;j=j+1;end"

Ok that's a stupid command but it's only to illustrate.

How does that command get executed? Does it get executed in parallel
with the work Scilab is currently doing, say while Scilab is busy
executing some (very clever) loop such as:

while %t;end

or does the ScilabEval-ed command interrupt the Scilab work?
or does it wait until the main flow (the while loop) has finished? In
that latter case, is the ScilabEval call blocking or did you keep a
queueing mechanism as it was the case so far (on this, see also below)?

A related topic: In which variable space does the ScilabEval-ed
command execute? Can the ScilabEval-ed command have access to the
variables that Scilab currently uses at the time of execution of the
ScilabEval? I.e. does it see the variables that can be seen from the
current pause (-n->) level?
Or does it execute in something more like a separate instance of
Scilab, that cannot access the variables from the main flow?

The latter would break a lot of features in Scipad, is it the case?


> The timer thread is quite dummy.
> It only send  periodically a "wake up" signal.
>
> "Wake up" signal are also send when asking for a TCL command through
> Scilab.
> A call to Tcl_EvalStr for instance.

In Scipad I had to use very complicated recursive combinations of
ScilabEval and TCL_EvalStr commands, for instance:

set comm "TCL_EvalStr(\"ScilabEval {TCL_EvalStr(\"\"set myvariable
false\"\",\"\"scipad\"\")} seq\",\"scipad\");"
ScilabEval $comm

Does this still work?


> - Some others commands may be broken.

Other aspects I can mention:

- All options of ScilabEval (see help ScilabEval) should be supported
in order not to break too many things in Scipad, and especially the
debugger. Is it the case?
You can also have a look at bug 2596, which is a drastically trimmed
down test case.


- The sciprompt variable set by tksynchro.
This variable is set by Scilab and used in Scipad every time Scipad
needs to ScilabEval something. The Scilab parse/run functions set
sciprompt to -1 while busy, and back to the pause level -n-> while idle.
Does this variable still exist (even if perhaps always set to zero)?
If it's not the case, then Scipad has to be modified, potentially in
depth. This is because in the debugger context Scipad has to know when
Scilab has finished executing the previous statement so that debug
commands can be issued. For that reason the debugger needs to know if
Scilab is still busy running or not.

And that's another reason why the ScilabEval options must be
supported. The debug commands are queued in such a way that they will
get executed as soon as Scilab stops at a breakpoint. If they now
execute in parallel with the main Scilab flow (my question above),
this will be an issue we'll have to fix. For instance retrieval of the
value of the watched variables at the breakpoint stop must be done
only after Scilab has finished the debug step, not in between.


> - As soon as we will have Windows similar functionalities, and taking
> into account your comments,
> we will include this into the Trunk and kindly ask you for some feedbacks.

I look forward to being able to test it. I guess most of my comments
above could be easier discussed against some reference implementation
made available in trunk.

Cheers,
Francois

Enrico Segre Enrico Segre
Reply | Threaded
Open this post in threaded view
|

Re: Tcl and Scilab Event loop

In reply to this post by Bruno JOFRET
Well, it looks like this is an important design issue, and it has to be
seen first of all in the context of design, not of a specific
implementation or presence of one given hidden variable or the other.
Solving it by ways of hackery for now will make more difficult to
address it later.

Multithreaded programming is a well estabilished paradigm, and concept
for letting different threads talk with each other, share or lock out
resources, synchronize, are well laid out. Signals, semaphores, queues,
priorities, determinism... and all that stuff.

In the case of Scipad, there are some actions, like scrolling existing
text, which are asynchronous with the computation engine, and other,
mainly those related to execution and debugging, which are entirely
dependent on its state, and should therefore made dependent on it in
tight way. If I think at scipad, I could list
-gui actions: the latency should be very low, of the order of 100msec
most, and the gui should get a high share of cpu - that will make the
program look more "responsive", therefore "refined"
-execute in scilab and debugging: it is entirely, state-machine
depending on the execution thread
-interpreter actions which can be executed asynchronously in a separate
namespace- for the moment I can think of the matlab conversion, eventual
lexing or profiling could fall in this cathegory too.

As for other tcl parts of scilab, "help" is a gui action with no return
impact on the computation engine, hence whatever makes it just look
faster is ok, BUT, "ged" is not only (we could discuss if merely zooming
a graphics or exporting it can be seen as a detached action or not, the
state of the graphic window can in principle influence an ongoing
execution), and "uicontrols" definitely not. Indeed, one of the
longstanding source of complaints was about synchronization issues of tk
GUIs - which usually people conoct in order to control some execution,
and which may include callbacks. If we say callbacks, their actions
should be predictable...
Now I understand (didn't try to compete with the noncompiling trunk
since a while, but saw the activity on svn) that uicontrols are being
ported to java. I don't know what the status is, but tcl or not, the
matters related to sharing and threading have to be addressed anyway.


Enrico


Enrico Segre Enrico Segre
Reply | Threaded
Open this post in threaded view
|

Re: Tcl and Scilab Event loop

In reply to this post by Bruno JOFRET
Well, it looks like this is an important design issue, and it has to be
seen first of all in the context of design, not of a specific
implementation or presence of one given hidden variable or the other.
Solving it by ways of hackery for now will make more difficult to
address it later.

Multithreaded programming is a well estabilished paradigm, and concept
for letting different threads talk with each other, share or lock out
resources, synchronize, are well laid out. Signals, semaphores, queues,
priorities, determinism... and all that stuff.

In the case of Scipad, there are some actions, like scrolling existing
text, which are asynchronous with the computation engine, and other,
mainly those related to execution and debugging, which are entirely
dependent on its state, and should therefore made dependent on it in
tight way. If I think at scipad, I could list
-gui actions: the latency should be very low, of the order of 100msec
most, and the gui should get a high share of cpu - that will make the
program look more "responsive", therefore "refined"
-execute in scilab and debugging: it is entirely, state-machine
depending on the execution thread
-interpreter actions which can be executed asynchronously in a separate
namespace- for the moment I can think of the matlab conversion, eventual
lexing or profiling could fall in this cathegory too.

As for other tcl parts of scilab, "help" is a gui action with no return
impact on the computation engine, hence whatever makes it just look
faster is ok, BUT, "ged" is not only (we could discuss if merely zooming
a graphics or exporting it can be seen as a detached action or not, the
state of the graphic window can in principle influence an ongoing
execution), and "uicontrols" definitely not. Indeed, one of the
longstanding source of complaints was about synchronization issues of tk
GUIs - which usually people conoct in order to control some execution,
and which may include callbacks. If we say callbacks, their actions
should be predictable...
Now I understand (didn't try to compete with the noncompiling trunk
since a while, but saw the activity on svn) that uicontrols are being
ported to java. I don't know what the status is, but tcl or not, the
matters related to sharing and threading have to be addressed anyway.


Enrico


Enrico Segre Enrico Segre
Reply | Threaded
Open this post in threaded view
|

Re: Tcl and Scilab Event loop

In reply to this post by Bruno JOFRET
Well, it looks like this is an important design issue, and it has to be
seen first of all in the context of design, not of a specific
implementation or presence of one given hidden variable or the other.
Solving it by ways of hackery for now will make more difficult to
address it later.

Multithreaded programming is a well estabilished paradigm, and concept
for letting different threads talk with each other, share or lock out
resources, synchronize, are well laid out. Signals, semaphores, queues,
priorities, determinism... and all that stuff.

In the case of Scipad, there are some actions, like scrolling existing
text, which are asynchronous with the computation engine, and other,
mainly those related to execution and debugging, which are entirely
dependent on its state, and should therefore made dependent on it in
tight way. If I think at scipad, I could list
-gui actions: the latency should be very low, of the order of 100msec
most, and the gui should get a high share of cpu - that will make the
program look more "responsive", therefore "refined"
-execute in scilab and debugging: it is entirely, state-machine
depending on the execution thread
-interpreter actions which can be executed asynchronously in a separate
namespace- for the moment I can think of the matlab conversion, eventual
lexing or profiling could fall in this cathegory too.

As for other tcl parts of scilab, "help" is a gui action with no return
impact on the computation engine, hence whatever makes it just look
faster is ok, BUT, "ged" is not only (we could discuss if merely zooming
a graphics or exporting it can be seen as a detached action or not, the
state of the graphic window can in principle influence an ongoing
execution), and "uicontrols" definitely not. Indeed, one of the
longstanding source of complaints was about synchronization issues of tk
GUIs - which usually people conoct in order to control some execution,
and which may include callbacks. If we say callbacks, their actions
should be predictable...
Now I understand (didn't try to compete with the noncompiling trunk
since a while, but saw the activity on svn) that uicontrols are being
ported to java. I don't know what the status is, but tcl or not, the
matters related to sharing and threading have to be addressed anyway.


Enrico


Francois Vogel-2 Francois Vogel-2
Reply | Threaded
Open this post in threaded view
|

Re: Tcl and Scilab Event loop

In reply to this post by Francois Vogel-2
Hi all,

It's now ten days since I have provided my thoughts about the message
Bruno Jofret sent on 11/01/08 about having the Tcl event loop in a
separate thread. There was also an answer from Enrico on that same topic.

Since then we have not heard from you on the Tcl/Tk topic. What do you
think about our feedback? Did you go ahead, cooking some wonderful
solution? Is the current implementation in trunk something you
consider final? Or did you perhaps just give up and plan to revert to
the previous event loop?

Could you please say what you intend to do, especially in the time
frame of "January 2008". This is indeed the advertised tentative
release date for Scilab 5 alpha on your site:
http://www.scilab.org/developers/index_developers.php?page=roadmap_5.0.

Thanks for an update.
Francois

Sylvestre Ledru Sylvestre Ledru
Reply | Threaded
Open this post in threaded view
|

Re: Tcl and Scilab Event loop

Hello,

Bruno is currently in holiday.
So, I will try to reply to your questions.
Bruno has a solution on his laptop which is better than the one in the
trunk. He will commit his work when he gets back.

Sylvestre

Le lundi 21 janvier 2008 à 21:48 +0100, François Vogel a écrit :

> Hi all,
>
> It's now ten days since I have provided my thoughts about the message
> Bruno Jofret sent on 11/01/08 about having the Tcl event loop in a
> separate thread. There was also an answer from Enrico on that same topic.
>
> Since then we have not heard from you on the Tcl/Tk topic. What do you
> think about our feedback? Did you go ahead, cooking some wonderful
> solution? Is the current implementation in trunk something you
> consider final? Or did you perhaps just give up and plan to revert to
> the previous event loop?
>
> Could you please say what you intend to do, especially in the time
> frame of "January 2008". This is indeed the advertised tentative
> release date for Scilab 5 alpha on your site:
> http://www.scilab.org/developers/index_developers.php?page=roadmap_5.0.
>
> Thanks for an update.
> Francois


Vincent COUVERT-3 Vincent COUVERT-3
Reply | Threaded
Open this post in threaded view
|

Re: Tcl and Scilab Event loop

In reply to this post by Francois Vogel-2
Hi François,

We are a little late for the developments of Scilab 5.0 Alpha version
functionalities.

This version will be available for week 7. We have a meeting tomorrow
morning to check that everything will be OK for this week and then we
will update Scilab web site.

Vincent

François Vogel a écrit :

> Hi all,
>
> It's now ten days since I have provided my thoughts about the message
> Bruno Jofret sent on 11/01/08 about having the Tcl event loop in a
> separate thread. There was also an answer from Enrico on that same topic.
>
> Since then we have not heard from you on the Tcl/Tk topic. What do you
> think about our feedback? Did you go ahead, cooking some wonderful
> solution? Is the current implementation in trunk something you
> consider final? Or did you perhaps just give up and plan to revert to
> the previous event loop?
>
> Could you please say what you intend to do, especially in the time
> frame of "January 2008". This is indeed the advertised tentative
> release date for Scilab 5 alpha on your site:
> http://www.scilab.org/developers/index_developers.php?page=roadmap_5.0.
>
> Thanks for an update.
> Francois
>
>

--
==============================================
Vincent COUVERT
Centre de Recherche INRIA Paris-Rocquencourt
Domaine de Voluceau - B.P. 105
78153 Le Chesnay Cedex
==============================================
Equipe Projet SCILAB
Bâtiment 1B - Bureau 013
Email : [hidden email]
Tél : +33 (0)1 39 63 54 46
Fax : +33 (0)1 39 63 55 94
==============================================

Bruno JOFRET Bruno JOFRET
Reply | Threaded
Open this post in threaded view
|

Re: Tcl and Scilab Event loop

In reply to this post by Francois Vogel-2
François Vogel wrote:

> Hi all,
>
> It's now ten days since I have provided my thoughts about the message
> Bruno Jofret sent on 11/01/08 about having the Tcl event loop in a
> separate thread. There was also an answer from Enrico on that same topic.
>
> Since then we have not heard from you on the Tcl/Tk topic. What do you
> think about our feedback? Did you go ahead, cooking some wonderful
> solution? Is the current implementation in trunk something you
> consider final? Or did you perhaps just give up and plan to revert to
> the previous event loop?
>
> Could you please say what you intend to do, especially in the time
> frame of "January 2008". This is indeed the advertised tentative
> release date for Scilab 5 alpha on your site:
> http://www.scilab.org/developers/index_developers.php?page=roadmap_5.0.
>
> Thanks for an update.
> Francois
>
>
Hi there,

It seems I did not make myself clear enough.
The version on the trunk is of course not a final one !
We are taking into account your pre requirements (bug 2514 and 2640) and
of course your previous mail.

It seems you kindly propose to test the first improvements, or I may
misunderstood...
Did you ever take some time to try our *"wonderful solution"* or did you
give up ???

For the time being I'm working on some issues that came out doing some
wonderful "while %t"
in scipad and launching it into Scilab.
I will also add some lock across Scilab "shared memory" (our beloved stack).

To crowed it all :
- If you were waiting my "go" to play with the solution, you can do it
- For what concerns the *"wonderful solution"*, I apologize not being a
wizard and go ahead
working on those threads and wait you test my first step.
- Each time I will make a step further to the "final solution" I'll post
on this ML exactly as I did last time.

Have a nice day.

--
Bruno JOFRET

         Project Engineer
___SCILAB - INRIA Rocquencourt___
  Domaine de Voluceau - B.P. 105
    78153 Le Chesnay Cedex
  Tel : (+33/0)1.39.63.58.63
  Mailto : [hidden email]
     http://www.scilab.org

Francois Vogel-2 Francois Vogel-2
Reply | Threaded
Open this post in threaded view
|

Re: Tcl and Scilab Event loop

Bruno JOFRET said on 28/01/2008 17:06:
> The version on the trunk is of course not a final one !

How the hell can I know that?

You ask for comments, you receive comments, following that I observe
the commits and see some activity around the Tcl topic, then nothing
for a week or so. How am I supposed to discover you were on holidays,
with a better version in your laptop?


> We are taking into account your pre requirements (bug 2514 and 2640) and
> of course your previous mail.

Good news.
 From what I could see so far you could have overlooked our feedback.I
  could never read an answer to our emails in which I had asked many
questions about what you intend to do. This does not help in receiving
further feedback later.


> It seems you kindly propose to test the first improvements, or I may
> misunderstood...

I have proposed it clearly before.
My email dated 11/01/08 ends with "I look forward to being able to
test it. I guess most of my comments above could be easier discussed
against some reference implementation made available in trunk.". Don't
know how to tell it clearer.


> Did you ever take some time to try our *"wonderful solution"* or did you
> give up ???

I did, for sure. And for free, remember. I did almost every day when
(from the commit logs) that day showed some activity on the topic.

Result: could never make anything work in the debugger, not a single
test case.

You want a random test case? OK, here is an oversimplified one:

Type or paste this in Scipad:

function foo
disp("first")
disp("second")
disp("third")
endfunction

Place a breakpoint (F9) on line containing disp("second") then
configure the debug (F10, click OK). And hit F11.

What should happen (and it happens in Scilab 4.1.2 with the same
version of Scipad) is that function foo gets executed with execution
stopping at the breakpointed line, this line gets highlighted, the
cursor doesn't keep on being the busy cursor, and so on.

You can easily check that this doesn't happen because on the Scilab
shell you should see (copy/pasted from Scilab 4.1.2):

  first
Stop after row     2 in function foo :

-->

For me this is broken (and always was in trunk since the beginning of
the java console in Scilab). Nothing moves in the Scilab shell.

Once the above case will be working, then you'll have a good piece done.
And we can then try step by step, run to cursor, break and other
debugger commands, watch variables and expressions, change them during
debug and a few other funny things perfectly working in the 4.x
environment.


> To crowed it all :
> - If you were waiting my "go" to play with the solution, you can do it

As said, I did before many times. No point in doing the same thing
once more with the same version of your changes.


> - For what concerns the *"wonderful solution"*, I apologize not being a
> wizard and go ahead
> working on those threads and wait you test my first step.

Done. I misunderstood you. Since nothing was working I couldn't think
you were waiting for me, I rather thought you had given up.


> - Each time I will make a step further to the "final solution" I'll post
> on this ML exactly as I did last time.

Perfect. Next time you'll ask for feedback I'll wait a couple of weeks
plus a reminder from you before answering.


> Have a nice day.

Have a nice night.

Francois


Enrico Segre Enrico Segre
Reply | Threaded
Open this post in threaded view
|

Re: Tcl and Scilab Event loop

In reply to this post by Bruno JOFRET
> Did you ever take some time to try our *"wonderful solution"* or did you
> give up ???

As for me, I simply can't compile the trunk any time I try - half hours of building process for just segfaults, libjvm.so not found, and whatever not. I'm simply fed up with trying wonderful solutions.
Enrico