[Scilab-users] Getting netCDF files into Scilab 6.0.1

classic Classic list List threaded Threaded
6 messages Options
arctica1963 arctica1963
Reply | Threaded
Open this post in threaded view
|

[Scilab-users] Getting netCDF files into Scilab 6.0.1

This post was updated on .
CONTENTS DELETED
The author has deleted this message.
Rafael Guerra Rafael Guerra
Reply | Threaded
Open this post in threaded view
|

Re: Getting netCDF files into Scilab 6.0.1

Hi,

You can strip the header rows and save a newfile.
Then use mfscanf as follows:

fid = mopen(newfile);
M = zeros(nrows, ncols);
for k=1:nrows
           M(k,:) = mfscanf(ncols,fid,"%f");
end

Regards,
Rafael

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Samuel GOUGEON Samuel GOUGEON
Reply | Threaded
Open this post in threaded view
|

Re: Getting netCDF files into Scilab 6.0.1

In reply to this post by arctica1963
Hello Lester,

Le 22/07/2018 à 11:12, arctica1963 a écrit :

> Hello,
>
> I am looking to use netCDF files generated from the GMT software in Scilab
> 6.0.1, but there is no easy way to work the problem other than exporting the
> data out from the binary grids to either XYZ or ESRI Ascii raster. The
> latter is a more compact form with the basic structure of a header
> describing the data limits and structure:
>
> ncols 1200
> nrows 1200
> xllcorner -10 Lower-left X (Long-Lat limits -10/10/-10/10
> yllcorner -10 Lower-Left Y
> cellsize 0.0166666666667 - increment = 1 arc minute
> nodata_value -9999
> 451.343170166 436.005554199 443.061035156 443.665924072 465.607574463
> 492.191741943 476.50994873 452.265014648 451.439880371 461.659393311 ......
> ...
> ... all of the Z-values
> ...
>
> -4323.2890625 -4315.12451172 -4307.19384766 -4305.27490234 -4311.49902344
> -4317.90771484 -4324.23046875 -4320.47070313 -4308.60107422 -4292.62011719
> -4280.39697266
>
> This is a moderately large file ~20 mB or so. Is there a way to reliably
> read this structure and vectorise X, Y, and Z? The data I am looking at is a
> grid so want to work on the whole grid as eith DEM data or gravity etc.
>
> As a test, I converted the grid to xyz, and tried csvRead, having replaced
> the spaces (which were not a constant number of spaces between values) with
> commas. The file was in excess of 50 mb, so not that huge really, but
> csvRead always failed to read all the file and drop out at one specific line
> (~7608) with a message of inconsistent column numbers, i.e. it though there
> were only 2 columns when in reality the whole thing was 3 (checked in the
> program Surfer). When one looks at the line specified, it is clearly 3
> columns.

Do you really get this error with the new nightly built Scilab release
http://www.scilab.org/en/development/nightly_builds/branch60
indicated by Antoine E. yesterday?
It is due to the same mgetl() bug (for Windows), that spreads all over
Scilab 6.0.1.

The continuous build
https://build.scilab.org/view/Scilab%206.0/job/scilab-6.0-windows-64/
can also be used without waiting for any publication delay.

Samuel

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
Samuel GOUGEON Samuel GOUGEON
Reply | Threaded
Open this post in threaded view
|

Re: Getting netCDF files into Scilab 6.0.1

Le 22/07/2018 à 11:45, Samuel Gougeon a écrit :

> Hello Lester,
>
> Le 22/07/2018 à 11:12, arctica1963 a écrit :
>> Hello,
>>
>> I am looking to use netCDF files generated from the GMT software in
>> Scilab
>> 6.0.1, but there is no easy way to work the problem other than
>> exporting the
>> data out from the binary grids to either XYZ or ESRI Ascii raster. The
>> latter is a more compact form with the basic structure of a header
>> describing the data limits and structure:
>>
>> ncols 1200
>> nrows 1200
>> xllcorner -10 Lower-left X (Long-Lat limits -10/10/-10/10
>> yllcorner -10 Lower-Left Y
>> cellsize 0.0166666666667 - increment = 1 arc minute
>> nodata_value -9999
>> 451.343170166    436.005554199    443.061035156 443.665924072    
>> 465.607574463
>> 492.191741943    476.50994873    452.265014648 451.439880371    
>> 461.659393311    ......
>> ...
>> ... all of the Z-values
>> ...
>>
>> -4323.2890625    -4315.12451172    -4307.19384766 -4305.27490234    
>> -4311.49902344
>> -4317.90771484    -4324.23046875    -4320.47070313 -4308.60107422    
>> -4292.62011719
>> -4280.39697266
>>
>> This is a moderately large file ~20 mB or so. Is there a way to reliably
>> read this structure and vectorise X, Y, and Z? The data I am looking
>> at is a
>> grid so want to work on the whole grid as eith DEM data or gravity etc.
>>
>> As a test, I converted the grid to xyz, and tried csvRead, having
>> replaced
>> the spaces (which were not a constant number of spaces between
>> values) with
>> commas. The file was in excess of 50 mb, so not that huge really, but
>> csvRead always failed to read all the file and drop out at one
>> specific line
>> (~7608) with a message of inconsistent column numbers, i.e. it though
>> there
>> were only 2 columns when in reality the whole thing was 3 (checked in
>> the
>> program Surfer). When one looks at the line specified, it is clearly 3
>> columns.
>
> Do you really get this error with the new nightly built Scilab release
> http://www.scilab.org/en/development/nightly_builds/branch60
> indicated by Antoine E. yesterday?
> It is due to the same mgetl() bug (for Windows), that spreads all over
> Scilab 6.0.1.
>
> The continuous build
> https://build.scilab.org/view/Scilab%206.0/job/scilab-6.0-windows-64/
> can also be used without waiting for any publication delay.

To me, the major advantage of the continuous built vs the nightly built
is that,
beside there is no delay, its SCIHOME directory and configuration files
are stable:
the version can be updated as often as required. Each time you get back
the config,
history, Scinotes config, etc as you left them from the previous version.

_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
arctica1963 arctica1963
Reply | Threaded
Open this post in threaded view
|

Re: Getting netCDF files into Scilab 6.0.1

Thanks for the ideas, I will check the new builds and see what happens.



--
Sent from: http://mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users
arctica1963 arctica1963
Reply | Threaded
Open this post in threaded view
|

Re: Getting netCDF files into Scilab 6.0.1

Hi all,

Just a quick update. Used a function to import an ESRI ascii raster seems to
work ok:

function [X,Y,Z]=asc_2_xyz(filename)
   
    fd=mopen(filename,'r'); // open ascii raster for read only
    k=mfscanf(fd,'%s\t%d'); nc=k(2); // number of columns
    k=mfscanf(fd,'%s\t%d'); nr=k(2); // number of rows
    k=mfscanf(fd,'%s\t%f'); x1=k(2); // X coordinate = lower left
    k=mfscanf(fd,'%s\t%f'); y1=k(2); // Y coordinate = lower left
    k=mfscanf(fd,'%s\t%f'); dxy=k(2); // X and Y increment
    k=mfscanf(fd,'%s\t%f'); nodata=k(2); // No data value (e.g. -9999)
    mclose(fd)

// x1 and y1 = minimum X and Y

    x2 = x1+dxy*nc; // Define maximum X coordinate
    y2 = y1+dxy*nr; // Define maximum Y coordinate

    Z=fscanfMat(filename); // Z matrix
    Z(Z==nodata)=%nan; // if any 'nodata' values in grid set to internal NaN
value
    Z=flipdim(Z,1); // for some reason the data are flipped on Y on import
so needed flipping

    xx=linspace(x1,x2,nc); // same number of columns but increment different
from dxy
    yy=linspace(y1,y2,nr);
    [X,Y]=meshgrid(xx,yy); // Coordinate matrices
   
endfunction

All looks fine, but the increment from linspace cannot match dxy unless one
has nc+1. Not a big issue as this can be corrected in GMT. But also reversed
the thing to output .asc and that gets all the dimensions and increment
fine:

function xyz_2_asc(filename,X,Y,Z,nodata,n)
// n = precision (e.g. 6)
// nodata should be appropriate for the Z data range
//e.g. xyz_2_asc(test.asc,x,y,z,-9999,6)
   
// Define matrix dimensions
    dimx=size(X); // 2-D
    dimy=size(Y);
    dimz=size(Z);
   
// Grid properties
    minx=min(X); //initial X coordinate (Lower left)
    maxx=max(X);
    miny=min(Y); //initial Y coordinate (Lower left)
    maxy=max(Y);
   
    dxy=abs(maxx-minx)/dimx(1) // correct for increment precision issues
   
// Z matrix
    Z(isnan(Z))=nodata; // add nan values (where present)) to output file
    Z=flipdim(Z,1); // invert matrix rows in Y
   
// Output file
   
    fd=mopen(filename,'w'); // open file for write
// Create ESRI Ascii raster header
//
// ncols 120
// nrows 120
// xllcorner -5
// yllcorner -5
// cellsize 0.0166666666667 - must be square pixels for ESRI ascii raster
// nodata_value -9999 (select value appropriate for dataset)
//
    mfprintf(fd,'ncols\t');
    mfprintf(fd,'%d\n',dimz(2));
    mfprintf(fd,'nrows\t');
    mfprintf(fd,'%d\n',dimz(1));
    mfprintf(fd,'xllcorner\t');
    mfprintf(fd,'%f\n',minx);
    mfprintf(fd,'yllcorner\t');
    mfprintf(fd,'%f\n',miny);
    mfprintf(fd,'cellsize\t');
    mfprintf(fd,'%.10f\n',dxy); // rounding errors > %.10f but not critical
to output
    mfprintf(fd,'NODATA_value\t');
    mfprintf(fd,%f\n',nodata);

// write Z values to file
       
    for i=1:dimz(1)
        for j=1:dimz(2)
            if j==dimz(2)
                mfprintf(fd,'%f\n',Z(i,j));
            else
                mfprintf(fd,'%f\t',Z(i,j));
            end
        end
    end
    mclose(fd);
       
endfunction

Ideally, it would be good to work out a way of reading the binary netCDF
grids, much smaller size, but at least it seems to work albeit a bit quirky
with the increments and flipping!

Lester



--
Sent from: http://mailinglists.scilab.org/Scilab-users-Mailing-Lists-Archives-f2602246.html
_______________________________________________
users mailing list
[hidden email]
http://lists.scilab.org/mailman/listinfo/users