[Scilab-users] Read file (.txt)

classic Classic list List threaded Threaded
13 messages Options
Daniel Stringari Daniel Stringari
Reply | Threaded
Open this post in threaded view
|

[Scilab-users] Read file (.txt)

Good evening everyone,

I'm integrating scilab with other software, so I need to read an output file
(.txt), like the one attached (output).

It is worth mentioning that this file does not always have the same
dimensions (matrix), so I am trying to find something robust that can deal
with this particularity. I tried to use the 'csvread' function and the
'read' function, but I still haven't been successful ... Any suggestions?

output.txt <http://mailinglists.scilab.org/file/t498028/output.txt>  





--
Sent from: http://mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Antoine ELIAS-2 Antoine ELIAS-2
Reply | Threaded
Open this post in threaded view
|

Re: Read file (.txt)

Hello Daniel,

Do you have try with "evstr" function ?

data = evstr(mgetl("output.txt"));

Regards,
Antoine

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
JLan JLan
Reply | Threaded
Open this post in threaded view
|

Re: Read file (.txt)

In reply to this post by Daniel Stringari

Hi Daniel

I made my own csvread, it seems to read your file OK with

exec('JcsvRead3.sce', -1);

M=JcsvRead3('output.txt','   ');

Brgds

Jan

On 2020-09-10 0:03 AM, Daniel Stringari wrote:
Good evening everyone,

I'm integrating scilab with other software, so I need to read an output file
(.txt), like the one attached (output).

It is worth mentioning that this file does not always have the same
dimensions (matrix), so I am trying to find something robust that can deal
with this particularity. I tried to use the 'csvread' function and the
'read' function, but I still haven't been successful ... Any suggestions?

output.txt <http://mailinglists.scilab.org/file/t498028/output.txt>  





--
Sent from: http://mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users


_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users

JcsvRead3.sce (2K) Download Attachment
der_Phil der_Phil
Reply | Threaded
Open this post in threaded view
|

Re: Read file (.txt)

In reply to this post by Antoine ELIAS-2
I think csvRead is just fine.

[data header] = csvRead(fname, ' ', '.', 'string', [], [], [], 0);

As you may notice the data Matrix contains a lot of empty columns.

This is due to the many white spaces in the text file

anyways, after reading the dta in you may use evstr() to convert into doubles.


BR
Philipp


Am Do., 10. Sept. 2020 um 04:03 Uhr schrieb <[hidden email]>:
Hello Daniel,

Do you have try with "evstr" function ?

data = evstr(mgetl("output.txt"));

Regards,
Antoine

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
JLan JLan
Reply | Threaded
Open this post in threaded view
|

Re: Read file (.txt)

The main problem I ran into with csvRead was reading files with a variable number of separators in each row. Then my suggested solution pads with 0. It could be changed to pad with NaN also.

Using strsplit in each row allows for several separators in the same conversion.  It may still be useful to use evstr row by row instead, if the number of spaces between numbers vary.

J


On 2020-09-10 10:46 AM, P M wrote:
I think csvRead is just fine.

[data header] = csvRead(fname, ' ', '.', 'string', [], [], [], 0);

As you may notice the data Matrix contains a lot of empty columns.

This is due to the many white spaces in the text file

anyways, after reading the dta in you may use evstr() to convert into doubles.


BR
Philipp


Am Do., 10. Sept. 2020 um 04:03 Uhr schrieb <[hidden email]>:
Hello Daniel,

Do you have try with "evstr" function ?

data = evstr(mgetl("output.txt"));

Regards,
Antoine

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Samuel GOUGEON Samuel GOUGEON
Reply | Threaded
Open this post in threaded view
|

Re: Read file (.txt)

In reply to this post by Daniel Stringari
Hello Daniel,

In addition to Antoine simple and efficient solution, the following works as well (and may be a bit faster for big files. To be confirmed):
File = "output.txt";
m = read(File, -1, size(evstr(mgetl(File,1)),2));

Regards


Le 10/09/2020 à 00:03, Daniel Stringari a écrit :
Good evening everyone,

I'm integrating scilab with other software, so I need to read an output file
(.txt), like the one attached (output).

It is worth mentioning that this file does not always have the same
dimensions (matrix), so I am trying to find something robust that can deal
with this particularity. I tried to use the 'csvread' function and the
'read' function, but I still haven't been successful ... Any suggestions?

output.txt <http://mailinglists.scilab.org/file/t498028/output.txt>  


_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Stefan Du Rietz Stefan Du Rietz
Reply | Threaded
Open this post in threaded view
|

Re: Read file (.txt)

Hello Samuel,

read() never works OK with m = -1, see your own comment in Bug 15075!

This prevented me from upgrading to Scilab 6 until I found out how to do
it with fscanfMat(). But then I had to remove all blank lines in my
files (which I had to quickly find different parts of my data).

--> tic(), M = fscanfMat(myfile.txt); toc()
  ans  =
    0.125278

--> size(M)
  ans  =
    13702.   9.

--> tic(), M = mgetl(myfile.txt); toc()
  ans  =
    0.050891
--> tic(), M = evstr(M); toc()
  ans  =
    1.854611

evstr is slow with large text vectors, compared to file reading.

--> size(evstr(mgetl(myfile, 1)), 2)
  ans  =
    9.
--> tic(), M = read(myfile.txt, -1, 9); toc()
  ans  =
    17.418157

So you have to know the file size completely (and when do you?)!

--> tic(), M = read(myfile.txt", 13702, 9); toc()
  ans  =
    0.085636

Regards
Stefan


On 2020-09-10 22:42, Samuel Gougeon wrote:

> Hello Daniel,
>
> In addition to Antoine simple and efficient solution, the following
> works as well (and may be a bit faster for big files. To be confirmed):
>
> File  =  "output.txt";
> m  =  read(File,-1, size(evstr(mgetl(File,1)),2));
> Regards
>
>
> Le 10/09/2020 à 00:03, Daniel Stringari a écrit :
>> Good evening everyone,
>>
>> I'm integrating scilab with other software, so I need to read an output file
>> (.txt), like the one attached (output).
>>
>> It is worth mentioning that this file does not always have the same
>> dimensions (matrix), so I am trying to find something robust that can deal
>> with this particularity. I tried to use the 'csvread' function and the
>> 'read' function, but I still haven't been successful ... Any suggestions?
>>
>> output.txt<http://mailinglists.scilab.org/file/t498028/output.txt>  
>
>
> _______________________________________________
> users mailing list
> [hidden email]
> http://lists.scilab.org/mailman/listinfo/users
>
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Stefan Du Rietz Stefan Du Rietz
Reply | Threaded
Open this post in threaded view
|

Re: Read file (.txt)

In reply to this post by Samuel GOUGEON
Please replace my latest e-mail with this.

Hello Samuel,

read() never works OK with m = -1, see your own comment in Bug 15075!

This prevented me from upgrading to Scilab 6 until I found out how to do
it with fscanfMat(). But then I had to remove all blank lines in my
files (which I had to quickly find different parts of my data).

--> tic(), M = fscanfMat(myfile); toc()
   ans  =
     0.125278

--> size(M)
   ans  =
     13702.   9.

--> tic(), M = mgetl(myfile); toc()
   ans  =
     0.050891
--> tic(), M = evstr(M); toc()
   ans  =
     1.854611

evstr is slow with large text vectors, compared to file reading.

--> size(evstr(mgetl(myfile, 1)), 2)
   ans  =
     9.
--> tic(), M = read(myfile, -1, 9); toc()
   ans  =
     17.418157

So you have to know the file size completely (and when do you?)!

--> tic(), M = read(myfile", 13702, 9); toc()
   ans  =
     0.085636

Regards
Stefan


On 2020-09-10 22:42, Samuel Gougeon wrote:

> Hello Daniel,
>
> In addition to Antoine simple and efficient solution, the following
> works as well (and may be a bit faster for big files. To be confirmed):
>
> File  =  "output.txt";
> m  =  read(File,-1, size(evstr(mgetl(File,1)),2));
> Regards
>
>
> Le 10/09/2020 à 00:03, Daniel Stringari a écrit :
>> Good evening everyone,
>>
>> I'm integrating scilab with other software, so I need to read an output file
>> (.txt), like the one attached (output).
>>
>> It is worth mentioning that this file does not always have the same
>> dimensions (matrix), so I am trying to find something robust that can deal
>> with this particularity. I tried to use the 'csvread' function and the
>> 'read' function, but I still haven't been successful ... Any suggestions?
>>
>> output.txt<http://mailinglists.scilab.org/file/t498028/output.txt>  
>
>
> _______________________________________________
> users mailing list
> [hidden email]
> http://lists.scilab.org/mailman/listinfo/users
>
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Samuel GOUGEON Samuel GOUGEON
Reply | Threaded
Open this post in threaded view
|

Re: Read file (.txt)

In reply to this post by Stefan Du Rietz
Le 11/09/2020 à 16:51, Stefan Du Rietz a écrit :
> Hello Samuel,
>
> read() never works OK with m = -1, see your own comment in Bug 15075!
>
> This prevented me from upgrading to Scilab 6 until I found out how to
> do it with fscanfMat(). But then I had to remove all blank lines in my
> files (which I had to quickly find different parts of my data).


For me, it perfectly works for Daniel's file, with 6.0.2 as well as with
6.1.0.
May be the 15075's report could be retested and updated.


_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Stefan Du Rietz Stefan Du Rietz
Reply | Threaded
Open this post in threaded view
|

Re: Read file (.txt)

On 2020-09-11 19:32, Samuel Gougeon wrote:

> Le 11/09/2020 à 16:51, Stefan Du Rietz a écrit :
>> Hello Samuel,
>>
>> read() never works OK with m = -1, see your own comment in Bug 15075!
>>
>> This prevented me from upgrading to Scilab 6 until I found out how to
>> do it with fscanfMat(). But then I had to remove all blank lines in my
>> files (which I had to quickly find different parts of my data).
>
>
> For me, it perfectly works for Daniel's file, with 6.0.2 as well as with
> 6.1.0.
> May be the 15075's report could be retested and updated.


With which OS? I am only using Linux (Ubuntu).

As a workaround, in Linux, you can quickly get the number of lines:
--> m = evstr(strtok(unix_g("wc -l " + filename), " "));

I will add to the bug report.

/Stefan
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Daniel Stringari Daniel Stringari
Reply | Threaded
Open this post in threaded view
|

Re: Read file (.txt)

Good evening everyone,

I appreciate everyone's availability. Great solutions!



--
Sent from: http://mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Stefan Du Rietz Stefan Du Rietz
Reply | Threaded
Open this post in threaded view
|

Re: Read file (.txt)

In reply to this post by Stefan Du Rietz


On 2020-09-12 00:06, Stefan Du Rietz wrote:

> On 2020-09-11 19:32, Samuel Gougeon wrote:
>> Le 11/09/2020 à 16:51, Stefan Du Rietz a écrit :
>>> Hello Samuel,
>>>
>>> read() never works OK with m = -1, see your own comment in Bug 15075!
>>>
>>> This prevented me from upgrading to Scilab 6 until I found out how to
>>> do it with fscanfMat(). But then I had to remove all blank lines in
>>> my files (which I had to quickly find different parts of my data).
>>
>>
>> For me, it perfectly works for Daniel's file, with 6.0.2 as well as
>> with 6.1.0.
>> May be the 15075's report could be retested and updated.
>
>
> With which OS? I am only using Linux (Ubuntu).
>
> As a workaround, in Linux, you can quickly get the number of lines:
> --> m = evstr(strtok(unix_g("wc -l " + filename), " "));
>
> I will add to the bug report.
>
> /Stefan


That does not work if the file contains empty lines because the argument
m to read() is the number of not empty lines to read! So one must
subtract the number of empty lines from m above ...

Why on earth was read() changed from the excellent previous version to
one of no use???
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Stefan Du Rietz Stefan Du Rietz
Reply | Threaded
Open this post in threaded view
|

Re: Read file (.txt)

On 2020-09-12 16:55, Stefan Du Rietz wrote:

>
>
> On 2020-09-12 00:06, Stefan Du Rietz wrote:
>> On 2020-09-11 19:32, Samuel Gougeon wrote:
>>> Le 11/09/2020 à 16:51, Stefan Du Rietz a écrit :
>>>> Hello Samuel,
>>>>
>>>> read() never works OK with m = -1, see your own comment in Bug 15075!
>>>>
>>>> This prevented me from upgrading to Scilab 6 until I found out how
>>>> to do it with fscanfMat(). But then I had to remove all blank lines
>>>> in my files (which I had to quickly find different parts of my data).
>>>
>>>
>>> For me, it perfectly works for Daniel's file, with 6.0.2 as well as
>>> with 6.1.0.
>>> May be the 15075's report could be retested and updated.
>>
>>
>> With which OS? I am only using Linux (Ubuntu).
>>
>> As a workaround, in Linux, you can quickly get the number of lines:
>> --> m = evstr(strtok(unix_g("wc -l " + filename), " "));
>>
>> I will add to the bug report.
>>
>> /Stefan
>
>
> That does not work if the file contains empty lines because the argument
> m to read() is the number of not empty lines to read! So one must
> subtract the number of empty lines from m above ...
>
> Why on earth was read() changed from the excellent previous version to
> one of no use???


But this does work (m is the number of not empty lines) and is about as
fast:
--> m = evstr(unix_g("grep -vc ^$ " + filename));

So, together with
--> n = size(evstr(mgetl(filename, 1)), 2);
--> M = read(filename, m, n);

Daniel can get what he wants.

/Stefan
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users