Help in reading txt file: how to skip the title

classic Classic list List threaded Threaded
5 messages Options
Zhiming Qi Zhiming Qi
Reply | Threaded
Open this post in threaded view
|

Help in reading txt file: how to skip the title

Hi,
 
I have a file which includes both titles and digit values. How can I get the digit values out but not the title when using read in Scilab?
 
Attached is the data format:
 

RZWQM2-3D

*****************************

* Subplot Definitions *

*****************************

1

*****************************

* COLUMN LABELS *

*****************************

DAY

DEPTH (CM)

SOIL WATER CONTENT (VOL)

****************** DATA STARTS HERE *****************

274 1.000 0.2898

274 2.000 0.2919

274 4.000 0.2939

274 7.000 0.2990

274 10.00 0.3035

274 13.00 0.3130

274 16.00 0.3172

274 20.00 0.3201

274 24.00 0.3243

274 27.00 0.3272

274 30.00 0.3300

274 33.00 0.3210

274 36.00 0.3223

274 40.00 0.3236

274 44.00 0.3258

274 48.00 0.3273

274 53.00 0.3302

274 57.00 0.3345

274 60.00 0.3381

274 63.00 0.3331

Thanks much,
 
Zhiming

harishankar ramachandran harishankar ramachandran
Reply | Threaded
Open this post in threaded view
|

Re: Help in reading txt file: how to skip the title

The following program does the specific job:

// to read in a data file containing header lines starting with
// "*" and to read in the columns after the last header line.
A=read("data.txt",-1,1,"(a)"); // read in the file
jj=grep(A,"*"); // find lines containing "*"

// We want to skip upto all lines containing "*" So start after
// the last such line
ibeg=jj(length(jj))+1;
iend=size(A,"r");
m=iend-ibeg+1;
[n,v1,v2,v3]=msscanf(m,A(ibeg:iend),"%f %f %f");

clf
subplot(3,1,1);
plot2d(1:m,v1');
subplot(3,1,2);
plot2d(1:m,v2');
subplot(3,1,3);
plot2d(1:m,v3');

// Note that this script hardcoded the number of columns and does
// no error checking. Since column information is present in the
// file, one could also extract that info and construct the
// format string appropriately. Or use the logic in fscanfMat to directly
// figure out the number of columns.

Not sure this is quite what you wanted.

regards

hari ramachandran

On Wednesday 06 May 2009 21:49, Zhiming Qi wrote:

> *****************************
> * Subplot Definitions *
> *****************************
> 1
> *****************************
> * COLUMN LABELS *
> *****************************
> DAY
> DEPTH (CM)
> SOIL WATER CONTENT (VOL)
> ****************** DATA STARTS HERE *****************
> 274 1.000 0.2898
> 274 2.000 0.2919
> 274 4.000 0.2939
> 274 7.000 0.2990
> 274 10.00 0.3035
> 274 13.00 0.3130
> 274 16.00 0.3172
> 274 20.00 0.3201
> 274 24.00 0.3243
> 274 27.00 0.3272
> 274 30.00 0.3300
> 274 33.00 0.3210
> 274 36.00 0.3223
> 274 40.00 0.3236
> 274 44.00 0.3258
> 274 48.00 0.3273
> 274 53.00 0.3302
> 274 57.00 0.3345
> 274 60.00 0.3381
> 274 63.00 0.3331

--
Dr. Hari Ramachandran, Professor, 332B ESB, EE Dept, IIT-Madras
Interests: Nonlinear Optics, Nonlinear Waves, Plasma Physics, Particle
           Simulations, Computational Algorithms, Linux.
Off: 91-44-2257-4421                    Fax: 91-44-2257-0120
Res: 91-44-2663-1863             Home Email: [hidden email]

Giri Giri
Reply | Threaded
Open this post in threaded view
|

Re: Help in reading txt file: how to skip the title

harishankar ramachandran wrote:

> The following program does the specific job:
>
> // to read in a data file containing header lines starting with
> // "*" and to read in the columns after the last header line.
> A=read("data.txt",-1,1,"(a)"); // read in the file
> jj=grep(A,"*"); // find lines containing "*"
>
> // We want to skip upto all lines containing "*" So start after
> // the last such line
> ibeg=jj(length(jj))+1;
> iend=size(A,"r");
> m=iend-ibeg+1;
> [n,v1,v2,v3]=msscanf(m,A(ibeg:iend),"%f %f %f");
>
> clf
> subplot(3,1,1);
> plot2d(1:m,v1');
> subplot(3,1,2);
> plot2d(1:m,v2');
> subplot(3,1,3);
> plot2d(1:m,v3');
>
> // Note that this script hardcoded the number of columns and does
> // no error checking. Since column information is present in the
> // file, one could also extract that info and construct the
> // format string appropriately. Or use the logic in fscanfMat to directly
> // figure out the number of columns.
>
> Not sure this is quite what you wanted.
>
> regards
>
> hari ramachandran
>
> On Wednesday 06 May 2009 21:49, Zhiming Qi wrote:
> > *****************************
> > * Subplot Definitions *
> > *****************************
> > 1
> > *****************************
> > * COLUMN LABELS *
> > *****************************
> > DAY
> > DEPTH (CM)
> > SOIL WATER CONTENT (VOL)
> > ****************** DATA STARTS HERE *****************
> > 274 1.000 0.2898
> > 274 2.000 0.2919
> > 274 4.000 0.2939
> > 274 7.000 0.2990
> > 274 10.00 0.3035
> > 274 13.00 0.3130
> > 274 16.00 0.3172
> > 274 20.00 0.3201
> > 274 24.00 0.3243
> > 274 27.00 0.3272
> > 274 30.00 0.3300
> > 274 33.00 0.3210
> > 274 36.00 0.3223
> > 274 40.00 0.3236
> > 274 44.00 0.3258
> > 274 48.00 0.3273
> > 274 53.00 0.3302
> > 274 57.00 0.3345
> > 274 60.00 0.3381
> > 274 63.00 0.3331
>
> --
> Dr. Hari Ramachandran, Professor, 332B ESB, EE Dept, IIT-Madras
> Interests: Nonlinear Optics, Nonlinear Waves, Plasma Physics, Particle
>            Simulations, Computational Algorithms, Linux.
> Off: 91-44-2257-4421                    Fax: 91-44-2257-0120
> Res: 91-44-2663-1863             Home Email: [hidden email]

Dear Prof. Ramachandran:
Thanks for the detailed solution, this is a common problem faced
by anyone trying process data obtained from lab instruments. I was
also planning to post some similar question as I am new user of
Scilab. Thanks to Mr Qi also!
Sincerely,
Giridhar

--
M. S. Giridhar
Thinfilms and Microsensors Group
Laboratory for Electro Optic Systems
Indian Space Research Organisation
1st Cross, 1st Stage, Peenya Industrial Estate,
Bangalore 560058.
Phone:2839 2294, 2839 2291 ext 2212



----------------------------------------------------------------
This mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. LEOS has taken every reasonable precaution to remove any viruses however you should also carry out your own virus checks before opening the e-mail or attachment.



Zhiming Qi Zhiming Qi
Reply | Threaded
Open this post in threaded view
|

Re: Re: [scilab-Users] Help in reading txt file: how to skip the title

Hi Hari,
Thanks much for your support. Attached is my reply to Prof. Ramachandran. Hope it could give you another perspective.
Best,
Zhiming
 
Dear Dr. Ramachandran,
 
Thank you very much! That's exactly what I want. The main problem is that I only know read('filename.txt', m, n) statement, and it could not work for data mixed with characters. I tried a lot and never find read statemtne like A=read("data.txt",-1,1,"(a)"). It helps a lot.
 
I kind of solved this problem by using file statement:
 
A =[]; B=[]; C=[];
v=file('open', 'data.txt', 'old');
for i = 1:36000
 if i<=12 then Head=fscanf(v, '%s');        // there is 12 headlines;
   else [AA BB CC]=fscanf(v, '%f %f %f');
        A=[A; AA]; B=[B; BB]; C=[C; CC];
end
end
// the data I want is
 [A, B, C]
 
Your read statement is so good that I do not have to write so long a program. I really appreciate your help.
Thanks much and have a great day,
 
Zhiming
Iowa State University
Agricultural Engineering Department
 
 
 
 
 
2009-05-12

isuzman

发件人: Giri
发送时间: 2009-05-08  09:32:31
收件人: [hidden email]
抄送:
主题: Re: [scilab-Users] Help in reading txt file: how to skip the title
harishankar ramachandran wrote:
 
> The following program does the specific job:
>
> // to read in a data file containing header lines starting with
> // "*" and to read in the columns after the last header line.
> A=read("data.txt",-1,1,"(a)"); // read in the file
> jj=grep(A,"*"); // find lines containing "*"
>
> // We want to skip upto all lines containing "*" So start after
> // the last such line
> ibeg=jj(length(jj))+1;
> iend=size(A,"r");
> m=iend-ibeg+1;
> [n,v1,v2,v3]=msscanf(m,A(ibeg:iend),"%f %f %f");
>
> clf
> subplot(3,1,1);
> plot2d(1:m,v1');
> subplot(3,1,2);
> plot2d(1:m,v2');
> subplot(3,1,3);
> plot2d(1:m,v3');
>
> // Note that this script hardcoded the number of columns and does
> // no error checking. Since column information is present in the
> // file, one could also extract that info and construct the
> // format string appropriately. Or use the logic in fscanfMat to directly
> // figure out the number of columns.
>
> Not sure this is quite what you wanted.
>
> regards
>
> hari ramachandran
>
> On Wednesday 06 May 2009 21:49, Zhiming Qi wrote:
>  > *****************************
>  > * Subplot Definitions *
>  > *****************************
>  > 1
>  > *****************************
>  > * COLUMN LABELS *
>  > *****************************
>  > DAY
>  > DEPTH (CM)
>  > SOIL WATER CONTENT (VOL)
>  > ****************** DATA STARTS HERE *****************
>  > 274 1.000 0.2898
>  > 274 2.000 0.2919
>  > 274 4.000 0.2939
>  > 274 7.000 0.2990
>  > 274 10.00 0.3035
>  > 274 13.00 0.3130
>  > 274 16.00 0.3172
>  > 274 20.00 0.3201
>  > 274 24.00 0.3243
>  > 274 27.00 0.3272
>  > 274 30.00 0.3300
>  > 274 33.00 0.3210
>  > 274 36.00 0.3223
>  > 274 40.00 0.3236
>  > 274 44.00 0.3258
>  > 274 48.00 0.3273
>  > 274 53.00 0.3302
>  > 274 57.00 0.3345
>  > 274 60.00 0.3381
>  > 274 63.00 0.3331
>
> --
> Dr. Hari Ramachandran, Professor, 332B ESB, EE Dept, IIT-Madras
> Interests: Nonlinear Optics, Nonlinear Waves, Plasma Physics, Particle
>            Simulations, Computational Algorithms, Linux.
> Off: 91-44-2257-4421                    Fax: 91-44-2257-0120
> Res: 91-44-2663-1863             Home Email: [hidden email]
 
Dear Prof. Ramachandran:
Thanks for the detailed solution, this is a common problem faced
by anyone trying process data obtained from lab instruments. I was
also planning to post some similar question as I am new user of
Scilab. Thanks to Mr Qi also!
Sincerely,
Giridhar
 
--
M. S. Giridhar
Thinfilms and Microsensors Group
Laboratory for Electro Optic Systems
Indian Space Research Organisation
1st Cross, 1st Stage, Peenya Industrial Estate,
Bangalore 560058.
Phone:2839 2294, 2839 2291 ext 2212
 
 
 
----------------------------------------------------------------
This mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. LEOS has taken every reasonable precaution to remove any viruses however you should also carry out your own virus checks before opening the e-mail or attachment. 
 
 
Osvaldo Carvalho Osvaldo Carvalho
Reply | Threaded
Open this post in threaded view
|

Re: Re: [scilab-Users] Help in reading txt file: how to skip the title

Hello,

Did you try the fscanfMat function to read the file? The following two lines seems to do the job, if I really got the question:

f = xgetfile() // locate the file
m = fscanfMat(f) // read the file over matrix m

You get

m =

274. 1. 0.2898
274. 2. 0.2919
274. 4. 0.2939
274. 7. 0.299
274. 10. 0.3035
274. 13. 0.313
274. 16. 0.3172
274. 20. 0.3201
274. 24. 0.3243
274. 27. 0.3272
274. 30. 0.33
274. 33. 0.321
274. 36. 0.3223
274. 40. 0.3236
274. 44. 0.3258
274. 48. 0.3273
274. 53. 0.3302
274. 57. 0.3345
274. 60. 0.3381
274. 63. 0.3331

The input file I used is (I supressed a line begining with a "1"; header lines should not start with a number):

*****************************
* Subplot Definitions *
*****************************

*****************************
* COLUMN LABELS *
*****************************
*****************************
DAY
DEPTH (CM)
SOIL WATER CONTENT (VOL)
****************** DATA STARTS HERE *****************
274 1.000 0.2898
274 2.000 0.2919
274 4.000 0.2939
274 7.000 0.2990
274 10.00 0.3035
274 13.00 0.3130
274 16.00 0.3172
274 20.00 0.3201
274 24.00 0.3243
274 27.00 0.3272
274 30.00 0.3300
274 33.00 0.3210
274 36.00 0.3223
274 40.00 0.3236
274 44.00 0.3258
274 48.00 0.3273
274 53.00 0.3302
274 57.00 0.3345
274 60.00 0.3381
274 63.00 0.3331

Osvaldo

"isuzman" <[hidden email]> wrote on 13/05/2009 01:45:43:

> Hi Hari,

> Thanks much for your support. Attached is my reply to Prof.
> Ramachandran. Hope it could give you another perspective.

> Best,
> Zhiming
>  
> Dear Dr. Ramachandran,
>  
> Thank you very much! That's exactly what I want. The main problem is
> that I only know read('filename.txt', m, n) statement, and it could
> not work for data mixed with characters. I tried a lot and never
> find read statemtne like A=read("data.txt",-1,1,"(a)"). It helps a lot.

>  
> I kind of solved this problem by using file statement:
>  
> A =[]; B=[]; C=[];
> v=file('open', 'data.txt', 'old');
> for i = 1:36000
>  if i<=12 then Head=fscanf(v, '%s');        // there is 12 headlines;
>    else [AA BB CC]=fscanf(v, '%f %f %f');
>         A=[A; AA]; B=[B; BB]; C=[C; CC];
> end
> end

> // the data I want is
>  [A, B, C]
>  
> Your read statement is so good that I do not have to write so long a
> program. I really appreciate your help.

> Thanks much and have a great day,
>  
> Zhiming
> Iowa State University
> Agricultural Engineering Department
>  
>  
>  
>  
>  
> 2009-05-12
>
> isuzman

>
> 发件人: Giri

> 发送时间: 2009-05-08  09:32:31
> 收件人: [hidden email]
> 抄送:
> 主题: Re: [scilab-Users] Help in reading txt file: how to skip the title
> harishankar ramachandran wrote:
>  
> > The following program does the specific job:
> >
> > // to read in a data file containing header lines starting with
> > // "*" and to read in the columns after the last header line.
> > A=read("data.txt",-1,1,"(a)"); // read in the file
> > jj=grep(A,"*"); // find lines containing "*"
> >
> > // We want to skip upto all lines containing "*" So start after
> > // the last such line
> > ibeg=jj(length(jj))+1;
> > iend=size(A,"r");
> > m=iend-ibeg+1;
> > [n,v1,v2,v3]=msscanf(m,A(ibeg:iend),"%f %f %f");
> >
> > clf
> > subplot(3,1,1);
> > plot2d(1:m,v1');
> > subplot(3,1,2);
> > plot2d(1:m,v2');
> > subplot(3,1,3);
> > plot2d(1:m,v3');
> >
> > // Note that this script hardcoded the number of columns and does
> > // no error checking. Since column information is present in the
> > // file, one could also extract that info and construct the
> > // format string appropriately. Or use the logic in fscanfMat to directly
> > // figure out the number of columns.
> >
> > Not sure this is quite what you wanted.
> >
> > regards
> >
> > hari ramachandran
> >
> > On Wednesday 06 May 2009 21:49, Zhiming Qi wrote:
> >  > *****************************
> >  > * Subplot Definitions *
> >  > *****************************
> >  > 1
> >  > *****************************
> >  > * COLUMN LABELS *
> >  > *****************************
> >  > DAY
> >  > DEPTH (CM)
> >  > SOIL WATER CONTENT (VOL)
> >  > ****************** DATA STARTS HERE *****************
> >  > 274 1.000 0.2898
> >  > 274 2.000 0.2919
> >  > 274 4.000 0.2939
> >  > 274 7.000 0.2990
> >  > 274 10.00 0.3035
> >  > 274 13.00 0.3130
> >  > 274 16.00 0.3172
> >  > 274 20.00 0.3201
> >  > 274 24.00 0.3243
> >  > 274 27.00 0.3272
> >  > 274 30.00 0.3300
> >  > 274 33.00 0.3210
> >  > 274 36.00 0.3223
> >  > 274 40.00 0.3236
> >  > 274 44.00 0.3258
> >  > 274 48.00 0.3273
> >  > 274 53.00 0.3302
> >  > 274 57.00 0.3345
> >  > 274 60.00 0.3381
> >  > 274 63.00 0.3331
> >
> > --
> > Dr. Hari Ramachandran, Professor, 332B ESB, EE Dept, IIT-Madras
> > Interests: Nonlinear Optics, Nonlinear Waves, Plasma Physics, Particle
> >            Simulations, Computational Algorithms, Linux.
> > Off: 91-44-2257-4421                    Fax: 91-44-2257-0120
> > Res: 91-44-2663-1863             Home Email: [hidden email]
>  
> Dear Prof. Ramachandran:
> Thanks for the detailed solution, this is a common problem faced
> by anyone trying process data obtained from lab instruments. I was
> also planning to post some similar question as I am new user of
> Scilab. Thanks to Mr Qi also!
> Sincerely,
> Giridhar
>  
> --
> M. S. Giridhar
> Thinfilms and Microsensors Group
> Laboratory for Electro Optic Systems
> Indian Space Research Organisation
> 1st Cross, 1st Stage, Peenya Industrial Estate,
> Bangalore 560058.
> Phone:2839 2294, 2839 2291 ext 2212
>  
>  
>  
> ----------------------------------------------------------------
> This mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended
> solely for the use of the addressee(s). If you are not the intended
> recipient, please notify the sender by e-mail and delete the
> original message. LEOS has taken every reasonable precaution to
> remove any viruses however you should also carry out your own virus
> checks before opening the e-mail or attachment.

>  
>