How to extract matrix in a huge text file

classic Classic list List threaded Threaded
4 messages Options
Orbeman Orbeman
Reply | Threaded
Open this post in threaded view
|

How to extract matrix in a huge text file

This post was updated on .
Hello,

I try to optimise the extraction of a huge matrix mxn  of a text file. The syntax of the text is :

single text line
matrix 1xn
single text line
matrix mxn

Data are exported from Moldflow (polymer injection software) with a native patran format (.ele) like :

SCANDIUM Fiber orientation tensor on elements results
    92622   102184              0        0        6
   EleID          Txx          Tyy          Tzz          Txy          Txz          Tyz
   17542 +3.40330E-01 +3.30003E-01 +3.29667E-01 +7.10335E-05 +1.90178E-04 -1.56400E-04
   17546 +3.38631E-01 +3.30777E-01 +3.30592E-01 +4.99531E-05 +1.41874E-04 +1.10836E-04
  102183 +3.40153E-01 +3.30106E-01 +3.29742E-01 +6.99497E-05 +1.86002E-04 -1.99423E-04
   17553 +3.48919E-01 +3.26850E-01 +3.24231E-01 -2.52756E-05 +1.52858E-04 -1.58712E-04
  102184 +3.38454E-01 +3.30879E-01 +3.30667E-01 +4.88692E-05 +1.37698E-04 +6.78126E-05
   17544 +3.47692E-01 +3.27132E-01 +3.25176E-01 -1.53594E-04 -2.83984E-04 +3.80141E-04
...

The function fscanfMat work fine il y a delete (manualy) the matrix 1xn. It is a way to use this function without get an error with the presence of the 1xn matrix ?

At this this I read the file and I duplicate only the 4:m+3 ($=m+3) lines and I read the new file with a read(file_path,-1,n) instruction but it takes too much time (about 60 s !).

Thanks !

I hope I was understable because I've some difficulties to speak (and write) in english.
Mike Page Mike Page
Reply | Threaded
Open this post in threaded view
|

RE: How to extract matrix in a huge text file

Hi,

I don't know how fast it would be on your huge matrix, but the following
works for your small example.

f=mopen("C:\temp\tmp.txt");
Title = mgetl(f,1)
 Title  =

 SCANDIUM Fiber orientation tensor on elements results

Info = evstr(mgetl(f,1))
 Info  =

    92622.    102184.    0.    0.    6.

Head = mgetl(f,1)
 Head  =

 EleID          Txx          Tyy          Tzz          Txy          Txz Tyz

Mat = evstr(mgetl(f,-1))
 Mat  =

    17542.     0.34033     0.330003    0.329667    0.0000710    0.0001902  -
0.0001564
    17546.     0.338631    0.330777    0.330592    0.0000500    0.0001419
0.0001108
    102183.    0.340153    0.330106    0.329742    0.0000699    0.0001860  -
0.0001994
    17553.     0.348919    0.32685     0.324231  - 0.0000253    0.0001529  -
0.0001587
    102184.    0.338454    0.330879    0.330667    0.0000489    0.0001377
0.0000678
    17544.     0.347692    0.327132    0.325176  - 0.0001536  - 0.0002840
0.0003801

HTH,
Mike.


-----Original Message-----
From: Orbeman [mailto:[hidden email]]
Sent: 17 August 2012 06:54
To: [hidden email]
Subject: [scilab-Users] How to extract matrix in a huge text file


Hello,

I try to optimise the extraction of a huge matrix mxn  of a text file. The
syntax of the text is :

single text line
matrix 1xn
single text line
matrix mxn

Data are exported from Moldflow (polymer injection software) with a native
patran format (.ele) like :

SCANDIUM Fiber orientation tensor on elements results
    92622   102184              0        0        6
   EleID          Txx          Tyy          Tzz          Txy          Txz
Tyz
   17542 +3.40330E-01 +3.30003E-01 +3.29667E-01 +7.10335E-05 +1.90178E-04
-1.56400E-04
   17546 +3.38631E-01 +3.30777E-01 +3.30592E-01 +4.99531E-05 +1.41874E-04
+1.10836E-04
  102183 +3.40153E-01 +3.30106E-01 +3.29742E-01 +6.99497E-05 +1.86002E-04
-1.99423E-04
   17553 +3.48919E-01 +3.26850E-01 +3.24231E-01 -2.52756E-05 +1.52858E-04
-1.58712E-04
  102184 +3.38454E-01 +3.30879E-01 +3.30667E-01 +4.88692E-05 +1.37698E-04
+6.78126E-05
   17544 +3.47692E-01 +3.27132E-01 +3.25176E-01 -1.53594E-04 -2.83984E-04
+3.80141E-04
...

The function fscanfMat work fine il y a delete (manualy) the matrix 1xn. It
is a way to use this function without get an error with the presence of the
1xn matrix ?

At this this I read the file and I duplicate only the 3:$ line but it takes
too much time (about 60 s !).

Thanks !

I hope I was understable because I've some difficulties to speak (and write)
in english.



--
View this message in context:
http://mailinglists.scilab.org/How-to-extract-matrix-in-a-huge-text-file-tp4
024708.html
Sent from the Scilab users - Mailing Lists Archives mailing list archive at
Nabble.com.

--
To unsubscribe from this mailing-list, please send an empty mail to
[hidden email]
To check the archives of this mailing list, see
http://mailinglists.scilab.org/



-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.2197 / Virus Database: 2437/5204 - Release Date: 08/16/12


--
To unsubscribe from this mailing-list, please send an empty mail to
[hidden email]
To check the archives of this mailing list, see
http://mailinglists.scilab.org/

Orbeman Orbeman
Reply | Threaded
Open this post in threaded view
|

Re: How to extract matrix in a huge text file

In reply to this post by Orbeman
Thanks but I get an allocation memory (999) with the mgetl function.
Orbeman Orbeman
Reply | Threaded
Open this post in threaded view
|

Re: How to extract matrix in a huge text file

I've found a speed method "en bidouillant" with VBScript. In fact, I use some script for the Moldflow API. I have also program an inverse optimisation with Moldflow and of course Scilab.

//
function data=lecture_patran(path_file, var)
   
    //suppression des 3 premières lignes d'entête du format natif patran
    str_VBS=['Set oFSO = CreateObject('+ascii(34)+'Scripting.FileSystemObject'+ascii(34)+')';
             'oFSO.CopyFile '+ascii(34)+path_file+ascii(34)+', '+ascii(34)+'c:\file_patran_src.ele'+ascii(34);
             'Set file_scr = oFSO.OpenTextFile('+ascii(34)+'c:\file_patran_src.ele'+ascii(34)+', 1)';
             'Set file_new = oFSO.OpenTextFile('+ascii(34)+path_file+ascii(34)+', 2)';
             'k = 0';
             'Do Until file_scr.AtEndOfStream';
             '    str_line = file_scr.ReadLine';
             '    if k >=3 then';
             '        file_new.WriteLine str_line';
             '    end if';
             '    k = k +1';
             'Loop';
             'oFSO.DeleteFile '+ascii(34)+'c:\file_patran_src.ele'+ascii(34)+', True']
   
    fd=mopen('c:\script_lecture_patran.vbs', 'wt')
    mputl(str_VBS, fd)
    mclose(fd)
   
    host('cscript '+ascii(34)+'c:\script_lecture_patran.vbs'+ascii(34))
   
    deletefile('c:\script_lecture_patran.vbs')
   
    //nombre de colonnes à lignes (ordre du tenseur)
    select var
    case 'a' then nj=6
    else nj=1
    end
   
    data=read(path_file, -1, nj)
   
endfunction