Quantcast

no mail mode

classic Classic list List threaded Threaded
14 messages Options
e-letter e-letter
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

no mail mode

Readers,

There is no digest mode for this mailing list. Is it possible to
subscribe but receive no mail please?
e-letter e-letter
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: no mail mode

On 17/11/2011, e-letter <[hidden email]> wrote:
> Readers,
>
> There is no digest mode for this mailing list. Is it possible to
> subscribe but receive no mail please?
>

Sorry, please ignore.
sylvestre sylvestre
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: no mail mode

In reply to this post by e-letter
Le jeudi 17 novembre 2011 à 08:57 +0000, e-letter a écrit :
> Readers,
>
> There is no digest mode for this mailing list. Is it possible to
> subscribe but receive no mail please?
For those who are not familiar, the mailing list are also available
here:
http://mailinglists.scilab.org/
And can be used like a forum if you prefer...

Sylvestre


Schreckenbach Stephan Schreckenbach Stephan
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

AW: [scilab-Users] no mail mode


Hi,

I look for a test of saisonality in time series.
The time series might be instationary and nonlinear and the saisonality
/ oscillation might have a changing amplitude. Furthermore the
distribution
might be unknown as well.
I need something to test for significant saisonality without knowing /
estimating a (linear) model of the time series.

ideas I got so far: Chi Square Test for independency:
I could test for independence of saison and mean value of the data

Chi Square Test to test for different means of two data groups.
I could test for a difference of the mean between several seasons.

Any more or better ideas?

Thanks in advance, Stephan

Adrien Vogt-Schilb Adrien Vogt-Schilb
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AW: [scilab-Users] no mail mode

I don't knwo if that is usefull for you:

I had to test seasonality and i did it successfully with Principal component analyses

you may use the grocer atoms toolbox to do PCAs

there's a lot of documentation on the web: https://www.google.com/search?q=seasonality+principal+component+analyses&hl=en

On 17/11/2011 14:07, Schreckenbach Stephan wrote:
Hi,

I look for a test of saisonality in time series.
The time series might be instationary and nonlinear and the saisonality
/ oscillation might have a changing amplitude. Furthermore the
distribution
might be unknown as well. 
I need something to test for significant saisonality without knowing /
estimating a (linear) model of the time series.

ideas I got so far: Chi Square Test for independency:
I could test for independence of saison and mean value of the data

Chi Square Test to test for different means of two data groups.
I could test for a difference of the mean between several seasons.

Any more or better ideas?

Thanks in advance, Stephan



--
Adrien Vogt-Schilb (Cired)
Tel: (+33) 1 43 94 73 77
Schreckenbach Stephan Schreckenbach Stephan
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

saisonality in time series

In reply to this post by Schreckenbach Stephan

Hi,

I look for a test of saisonality in time series.
The time series might be instationary and nonlinear and the saisonality
/ oscillation might have a changing amplitude. Furthermore the
distribution
might be unknown as well.
I need something to test for significant saisonality without knowing /
estimating a (linear) model of the time series.

ideas I got so far: Chi Square Test for independency:
I could test for independence of saison and mean value of the data

Chi Square Test to test for different means of two data groups.
I could test for a difference of the mean between several seasons.

Any more or better ideas?

Thanks in advance, Stephan

Petter Wingren Petter Wingren
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: saisonality in time series

Did a quick search but couldnt find anything obvious. I suppose the
word you are looking for is seasonality - maybe that helps in finding
something useful.

On Thu, Nov 17, 2011 at 3:36 PM, Schreckenbach Stephan
<[hidden email]> wrote:

>
> Hi,
>
> I look for a test of saisonality in time series.
> The time series might be instationary and nonlinear and the saisonality
> / oscillation might have a changing amplitude. Furthermore the
> distribution
> might be unknown as well.
> I need something to test for significant saisonality without knowing /
> estimating a (linear) model of the time series.
>
> ideas I got so far: Chi Square Test for independency:
> I could test for independence of saison and mean value of the data
>
> Chi Square Test to test for different means of two data groups.
> I could test for a difference of the mean between several seasons.
>
> Any more or better ideas?
>
> Thanks in advance, Stephan
>
>
Mike Page Mike Page
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: saisonality in time series

Hi,

I don't know much about this application, but the Cepstrum can be used to
find hidden periodicity in time series.  Might be worth trying?  I have used
it for finding rotational components in the vibration signatures from
rotating machinery.  There's a simple example here
(http://www.dliengineering.com/downloads/cepstrum%20analysis.pdf).

Mike.


-----Original Message-----
From: Petter Wingren [mailto:[hidden email]]
Sent: 17 November 2011 17:18
To: [hidden email]
Subject: Re: [scilab-Users] saisonality in time series


Did a quick search but couldnt find anything obvious. I suppose the
word you are looking for is seasonality - maybe that helps in finding
something useful.

On Thu, Nov 17, 2011 at 3:36 PM, Schreckenbach Stephan
<[hidden email]> wrote:

>
> Hi,
>
> I look for a test of saisonality in time series.
> The time series might be instationary and nonlinear and the saisonality
> / oscillation might have a changing amplitude. Furthermore the
> distribution
> might be unknown as well.
> I need something to test for significant saisonality without knowing /
> estimating a (linear) model of the time series.
>
> ideas I got so far: Chi Square Test for independency:
> I could test for independence of saison and mean value of the data
>
> Chi Square Test to test for different means of two data groups.
> I could test for a difference of the mean between several seasons.
>
> Any more or better ideas?
>
> Thanks in advance, Stephan
>
>

Charles Warner Charles Warner
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: saisonality in time series

Although "seasonality" is not the term I use for long term trends hidden in noisy data, I have had some success by taking the log of the data, and running an FFT on the log data.  Usually, I have some prior knowledge of the long-term periodic trends I expect, so it is relatively easy to determine quickly if this method works.  Plotting the log of the data also gives one a good feel for whether the data is stationary, or whether there are windows of data that can be treated as stationary.  Any changing magnitude effect is, of course, reduced when on works with logs, but such effects can help one understand what the raw data is really telling you.

Charlie

On Thu, Nov 17, 2011 at 12:40 PM, Mike Page <[hidden email]> wrote:
Hi,

I don't know much about this application, but the Cepstrum can be used to
find hidden periodicity in time series.  Might be worth trying?  I have used
it for finding rotational components in the vibration signatures from
rotating machinery.  There's a simple example here
(http://www.dliengineering.com/downloads/cepstrum%20analysis.pdf).

Mike.


-----Original Message-----
From: Petter Wingren [mailto:[hidden email]]
Sent: 17 November 2011 17:18
To: [hidden email]
Subject: Re: [scilab-Users] saisonality in time series


Did a quick search but couldnt find anything obvious. I suppose the
word you are looking for is seasonality - maybe that helps in finding
something useful.

On Thu, Nov 17, 2011 at 3:36 PM, Schreckenbach Stephan
<[hidden email]> wrote:
>
> Hi,
>
> I look for a test of saisonality in time series.
> The time series might be instationary and nonlinear and the saisonality
> / oscillation might have a changing amplitude. Furthermore the
> distribution
> might be unknown as well.
> I need something to test for significant saisonality without knowing /
> estimating a (linear) model of the time series.
>
> ideas I got so far: Chi Square Test for independency:
> I could test for independence of saison and mean value of the data
>
> Chi Square Test to test for different means of two data groups.
> I could test for a difference of the mean between several seasons.
>
> Any more or better ideas?
>
> Thanks in advance, Stephan
>
>



Schreckenbach Stephan Schreckenbach Stephan
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

AW: [scilab-Users] saisonality in time series

Hi,

 

sorry, of course I meant seasonality.

The time series consists of longer term trends, short term noise and short time seasonality.

oscillations / seasonality, if any, it is most likely to be nonharmonic. I look for distinct frequencies.

When I did a FFT plot of the original time series there was noise only in the spectrum.

I will give it a run with the differenciated series / the log of the data.

There is still the question how to test for significance of the found seasonality.

 

Stephan

 

 


Von: Charles Warner [mailto:[hidden email]]
Gesendet: Freitag, 18. November 2011 00:34
An: [hidden email]
Betreff: Re: [scilab-Users] saisonality in time series

 

Although "seasonality" is not the term I use for long term trends hidden in noisy data, I have had some success by taking the log of the data, and running an FFT on the log data.  Usually, I have some prior knowledge of the long-term periodic trends I expect, so it is relatively easy to determine quickly if this method works.  Plotting the log of the data also gives one a good feel for whether the data is stationary, or whether there are windows of data that can be treated as stationary.  Any changing magnitude effect is, of course, reduced when on works with logs, but such effects can help one understand what the raw data is really telling you.

Charlie

On Thu, Nov 17, 2011 at 12:40 PM, Mike Page <[hidden email]> wrote:

Hi,

I don't know much about this application, but the Cepstrum can be used to
find hidden periodicity in time series.  Might be worth trying?  I have used
it for finding rotational components in the vibration signatures from
rotating machinery.  There's a simple example here
(http://www.dliengineering.com/downloads/cepstrum%20analysis.pdf).

Mike.



-----Original Message-----
From: Petter Wingren [mailto:[hidden email]]
Sent: 17 November 2011 17:18
To: [hidden email]
Subject: Re: [scilab-Users] saisonality in time series


Did a quick search but couldnt find anything obvious. I suppose the
word you are looking for is seasonality - maybe that helps in finding
something useful.

On Thu, Nov 17, 2011 at 3:36 PM, Schreckenbach Stephan
<[hidden email]> wrote:
>
> Hi,
>
> I look for a test of saisonality in time series.
> The time series might be instationary and nonlinear and the saisonality
> / oscillation might have a changing amplitude. Furthermore the
> distribution
> might be unknown as well.
> I need something to test for significant saisonality without knowing /
> estimating a (linear) model of the time series.
>
> ideas I got so far: Chi Square Test for independency:
> I could test for independence of saison and mean value of the data
>
> Chi Square Test to test for different means of two data groups.
> I could test for a difference of the mean between several seasons.
>
> Any more or better ideas?
>
> Thanks in advance, Stephan
>
>

 

Charles Warner Charles Warner
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: saisonality in time series

Another trick I have found that greatly reduces FFT noise it to temporarily mask any localized "spikes" in the data (such spikes, with a narrow temporal profile have a very broad spectral distribution).  One can also try to eliminate any offset by subtracting the mean (or the geometric mean or harmonic mean- the appropriate mean would be dictated by the nature of the data).  This should hopefully reduce the scale of the FFT amplitude, making it easier to spot any (especially low-frequency, or seasonal) potential frequency components.

On Fri, Nov 18, 2011 at 3:09 AM, Schreckenbach Stephan <[hidden email]> wrote:

Hi,

 

sorry, of course I meant seasonality.

The time series consists of longer term trends, short term noise and short time seasonality.

oscillations / seasonality, if any, it is most likely to be nonharmonic. I look for distinct frequencies.

When I did a FFT plot of the original time series there was noise only in the spectrum.

I will give it a run with the differenciated series / the log of the data.

There is still the question how to test for significance of the found seasonality.

 

Stephan

 

 


Von: Charles Warner [mailto:[hidden email]]
Gesendet: Freitag, 18. November 2011 00:34
An: [hidden email]
Betreff: Re: [scilab-Users] saisonality in time series

 

Although "seasonality" is not the term I use for long term trends hidden in noisy data, I have had some success by taking the log of the data, and running an FFT on the log data.  Usually, I have some prior knowledge of the long-term periodic trends I expect, so it is relatively easy to determine quickly if this method works.  Plotting the log of the data also gives one a good feel for whether the data is stationary, or whether there are windows of data that can be treated as stationary.  Any changing magnitude effect is, of course, reduced when on works with logs, but such effects can help one understand what the raw data is really telling you.

Charlie

On Thu, Nov 17, 2011 at 12:40 PM, Mike Page <[hidden email]> wrote:

Hi,

I don't know much about this application, but the Cepstrum can be used to
find hidden periodicity in time series.  Might be worth trying?  I have used
it for finding rotational components in the vibration signatures from
rotating machinery.  There's a simple example here
(http://www.dliengineering.com/downloads/cepstrum%20analysis.pdf).

Mike.



-----Original Message-----
From: Petter Wingren [mailto:[hidden email]]
Sent: 17 November 2011 17:18
To: [hidden email]
Subject: Re: [scilab-Users] saisonality in time series


Did a quick search but couldnt find anything obvious. I suppose the
word you are looking for is seasonality - maybe that helps in finding
something useful.

On Thu, Nov 17, 2011 at 3:36 PM, Schreckenbach Stephan
<[hidden email]> wrote:
>
> Hi,
>
> I look for a test of saisonality in time series.
> The time series might be instationary and nonlinear and the saisonality
> / oscillation might have a changing amplitude. Furthermore the
> distribution
> might be unknown as well.
> I need something to test for significant saisonality without knowing /
> estimating a (linear) model of the time series.
>
> ideas I got so far: Chi Square Test for independency:
> I could test for independence of saison and mean value of the data
>
> Chi Square Test to test for different means of two data groups.
> I could test for a difference of the mean between several seasons.
>
> Any more or better ideas?
>
> Thanks in advance, Stephan
>
>

 


Schreckenbach Stephan Schreckenbach Stephan
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

AW: [scilab-Users] saisonality in time series

Filtering temporal spikes is a good idea, since there are some of them. I will try that.

The data sample as around 7000 data points, the frequency I look for is around 1/10 * sample rate.

 

May be there are methods that are better suited for identifying frequency components in that kind of data?

FFT always describes the time series by harmonic oszillations, which might not work well

if oscillations are not (strictly) harmonic.

 

What about wavelets (don’t know much about it yet, though)?

 

 

Stephan

 

 


Von: Charles Warner [mailto:[hidden email]]
Gesendet: Samstag, 19. November 2011 05:12
An: [hidden email]
Betreff: Re: [scilab-Users] saisonality in time series

 

Another trick I have found that greatly reduces FFT noise it to temporarily mask any localized "spikes" in the data (such spikes, with a narrow temporal profile have a very broad spectral distribution).  One can also try to eliminate any offset by subtracting the mean (or the geometric mean or harmonic mean- the appropriate mean would be dictated by the nature of the data).  This should hopefully reduce the scale of the FFT amplitude, making it easier to spot any (especially low-frequency, or seasonal) potential frequency components.

On Fri, Nov 18, 2011 at 3:09 AM, Schreckenbach Stephan <[hidden email]> wrote:

Hi,

 

sorry, of course I meant seasonality.

The time series consists of longer term trends, short term noise and short time seasonality.

oscillations / seasonality, if any, it is most likely to be nonharmonic. I look for distinct frequencies.

When I did a FFT plot of the original time series there was noise only in the spectrum.

I will give it a run with the differenciated series / the log of the data.

There is still the question how to test for significance of the found seasonality.

 

Stephan

 

 


Von: Charles Warner [mailto:[hidden email]]
Gesendet: Freitag, 18. November 2011 00:34
An: [hidden email]
Betreff: Re: [scilab-Users] saisonality in time series

 

Although "seasonality" is not the term I use for long term trends hidden in noisy data, I have had some success by taking the log of the data, and running an FFT on the log data.  Usually, I have some prior knowledge of the long-term periodic trends I expect, so it is relatively easy to determine quickly if this method works.  Plotting the log of the data also gives one a good feel for whether the data is stationary, or whether there are windows of data that can be treated as stationary.  Any changing magnitude effect is, of course, reduced when on works with logs, but such effects can help one understand what the raw data is really telling you.

Charlie

On Thu, Nov 17, 2011 at 12:40 PM, Mike Page <[hidden email]> wrote:

Hi,

I don't know much about this application, but the Cepstrum can be used to
find hidden periodicity in time series.  Might be worth trying?  I have used
it for finding rotational components in the vibration signatures from
rotating machinery.  There's a simple example here
(http://www.dliengineering.com/downloads/cepstrum%20analysis.pdf).

Mike.



-----Original Message-----
From: Petter Wingren [mailto:[hidden email]]
Sent: 17 November 2011 17:18
To: [hidden email]
Subject: Re: [scilab-Users] saisonality in time series


Did a quick search but couldnt find anything obvious. I suppose the
word you are looking for is seasonality - maybe that helps in finding
something useful.

On Thu, Nov 17, 2011 at 3:36 PM, Schreckenbach Stephan
<[hidden email]> wrote:
>
> Hi,
>
> I look for a test of saisonality in time series.
> The time series might be instationary and nonlinear and the saisonality
> / oscillation might have a changing amplitude. Furthermore the
> distribution
> might be unknown as well.
> I need something to test for significant saisonality without knowing /
> estimating a (linear) model of the time series.
>
> ideas I got so far: Chi Square Test for independency:
> I could test for independence of saison and mean value of the data
>
> Chi Square Test to test for different means of two data groups.
> I could test for a difference of the mean between several seasons.
>
> Any more or better ideas?
>
> Thanks in advance, Stephan
>
>

 

 

Ginters Bušs Ginters Bušs
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: saisonality in time series

In reply to this post by Charles Warner
Better stick with DFT, smoothed DFT or try seasonal adjustment freeware Demetra+   - that's what official statisticians might do.

gin


On Mon, Nov 21, 2011 at 10:00 AM, Schreckenbach Stephan <[hidden email]> wrote:

Filtering temporal spikes is a good idea, since there are some of them. I will try that.

The data sample as around 7000 data points, the frequency I look for is around 1/10 * sample rate.

 

May be there are methods that are better suited for identifying frequency components in that kind of data?

FFT always describes the time series by harmonic oszillations, which might not work well

if oscillations are not (strictly) harmonic.

 

What about wavelets (don’t know much about it yet, though)?

 

 

Stephan

 

 


Von: Charles Warner [mailto:[hidden email]]
Gesendet: Samstag, 19. November 2011 05:12


An: [hidden email]
Betreff: Re: [scilab-Users] saisonality in time series

 

Another trick I have found that greatly reduces FFT noise it to temporarily mask any localized "spikes" in the data (such spikes, with a narrow temporal profile have a very broad spectral distribution).  One can also try to eliminate any offset by subtracting the mean (or the geometric mean or harmonic mean- the appropriate mean would be dictated by the nature of the data).  This should hopefully reduce the scale of the FFT amplitude, making it easier to spot any (especially low-frequency, or seasonal) potential frequency components.

On Fri, Nov 18, 2011 at 3:09 AM, Schreckenbach Stephan <[hidden email]> wrote:

Hi,

 

sorry, of course I meant seasonality.

The time series consists of longer term trends, short term noise and short time seasonality.

oscillations / seasonality, if any, it is most likely to be nonharmonic. I look for distinct frequencies.

When I did a FFT plot of the original time series there was noise only in the spectrum.

I will give it a run with the differenciated series / the log of the data.

There is still the question how to test for significance of the found seasonality.

 

Stephan

 

 


Von: Charles Warner [mailto:[hidden email]]
Gesendet: Freitag, 18. November 2011 00:34
An: [hidden email]
Betreff: Re: [scilab-Users] saisonality in time series

 

Although "seasonality" is not the term I use for long term trends hidden in noisy data, I have had some success by taking the log of the data, and running an FFT on the log data.  Usually, I have some prior knowledge of the long-term periodic trends I expect, so it is relatively easy to determine quickly if this method works.  Plotting the log of the data also gives one a good feel for whether the data is stationary, or whether there are windows of data that can be treated as stationary.  Any changing magnitude effect is, of course, reduced when on works with logs, but such effects can help one understand what the raw data is really telling you.

Charlie

On Thu, Nov 17, 2011 at 12:40 PM, Mike Page <[hidden email]> wrote:

Hi,

I don't know much about this application, but the Cepstrum can be used to
find hidden periodicity in time series.  Might be worth trying?  I have used
it for finding rotational components in the vibration signatures from
rotating machinery.  There's a simple example here
(http://www.dliengineering.com/downloads/cepstrum%20analysis.pdf).

Mike.



-----Original Message-----
From: Petter Wingren [mailto:[hidden email]]
Sent: 17 November 2011 17:18
To: [hidden email]
Subject: Re: [scilab-Users] saisonality in time series


Did a quick search but couldnt find anything obvious. I suppose the
word you are looking for is seasonality - maybe that helps in finding
something useful.

On Thu, Nov 17, 2011 at 3:36 PM, Schreckenbach Stephan
<[hidden email]> wrote:
>
> Hi,
>
> I look for a test of saisonality in time series.
> The time series might be instationary and nonlinear and the saisonality
> / oscillation might have a changing amplitude. Furthermore the
> distribution
> might be unknown as well.
> I need something to test for significant saisonality without knowing /
> estimating a (linear) model of the time series.
>
> ideas I got so far: Chi Square Test for independency:
> I could test for independence of saison and mean value of the data
>
> Chi Square Test to test for different means of two data groups.
> I could test for a difference of the mean between several seasons.
>
> Any more or better ideas?
>
> Thanks in advance, Stephan
>
>

 

 


Charles Warner Charles Warner
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: saisonality in time series

Stephan-
Sounds like you are working with data sets that resemble some that I work with frequently.  About the same size, non-stationary seasonality (which I prefer to call low-frequency periodic trends), high, non-stationary temporally variable "noisy" signals.  I usually have a priori knowledge that there is likely aliasing due to the fact that I am limited in sampling rate vs. total time period (varying the sampling rate for different collection instances can sometimes resolve at least part of this issue).  I haven't figured out how to get the aliasing out of the system, although there should be some way to do this based on "unfolding" the spectrum.

Anyway, the approach I take is this.  Starting with a COPY of the raw data, I run it through NIST Dataplot to get a feel for the data, using what they call the 4-plot.  This gives me a feel for the periodicity, auto-correlation, statistical distribution, and helps me identify the "ouliers" (i.e., spikes, 0's, any data point that is "unusual"), which are then filtered out by replacing specific data points with a "more reasonable" value (which is why I use a copy of the raw data, rather than the original).  Usually use a spreadsheet for this.  I might run the data through Dataplot a couple of times to evaluate the effects my "filtering" have had.  Usually, still in the spreadsheet, I next remove the "DC offset" by subtracting one of the Pythagorean means.  Which is appropriate depends on the nature of the data.   This reduces the low end of the spectrum, which is where the "trends" are located. 

For the Fourier analysis, I could stay with Dataplot, but I find it much easier to extract information from the Scilab approach.  Furthermore, Scilab offers an alternative DFT processcalled MESE.  The Maximum Entropy Spectral Estimate (MESE), designed to produce high-resolution, low-bias spectral estimate (refer to page 128 of the Signal processing With Scilab manual, or available here). MESE incorporates no information in the estimated spectrum about the autocorrelation lags. That is to say that the bias resulting from the leakage from the window sidelobes should be eliminated (or at least minimized in some sense).   In other words, one tends to get cleaner "spikes" in the spectrum.  It is much easier to pick out the lower-frequency components of the signals with this procedure.  One then subtracts these components from the working data, and repeats the process.  Ideally, you have extracted all of the available information from the data when the residual is Gaussian white noise (a point I have never actually reached in practice). 

I don't believe "windowing" techniques will work with low-frequency components, although I could be mistaken in this.  I have toyed with windowing when I have reduced my residual to what has the appearance of a frequency-modulated signal- but, then, I am looking to characterize such events.  The information you are trying to extract will ultimately dictate what approach you take.

I am attaching a "working document" that I have put together giving more detail on this approach.

Charlie

2011/11/21 Ginters Bušs <[hidden email]>
Better stick with DFT, smoothed DFT or try seasonal adjustment freeware Demetra+   - that's what official statisticians might do.

gin


On Mon, Nov 21, 2011 at 10:00 AM, Schreckenbach Stephan <[hidden email]> wrote:

Filtering temporal spikes is a good idea, since there are some of them. I will try that.

The data sample as around 7000 data points, the frequency I look for is around 1/10 * sample rate.

 

May be there are methods that are better suited for identifying frequency components in that kind of data?

FFT always describes the time series by harmonic oszillations, which might not work well

if oscillations are not (strictly) harmonic.

 

What about wavelets (don’t know much about it yet, though)?

 

 

Stephan

 

 


Von: Charles Warner [mailto:[hidden email]]

Gesendet: Samstag, 19. November 2011 05:12


An: [hidden email]
Betreff: Re: [scilab-Users] saisonality in time series

 

Another trick I have found that greatly reduces FFT noise it to temporarily mask any localized "spikes" in the data (such spikes, with a narrow temporal profile have a very broad spectral distribution).  One can also try to eliminate any offset by subtracting the mean (or the geometric mean or harmonic mean- the appropriate mean would be dictated by the nature of the data).  This should hopefully reduce the scale of the FFT amplitude, making it easier to spot any (especially low-frequency, or seasonal) potential frequency components.

On Fri, Nov 18, 2011 at 3:09 AM, Schreckenbach Stephan <[hidden email]> wrote:

Hi,

 

sorry, of course I meant seasonality.

The time series consists of longer term trends, short term noise and short time seasonality.

oscillations / seasonality, if any, it is most likely to be nonharmonic. I look for distinct frequencies.

When I did a FFT plot of the original time series there was noise only in the spectrum.

I will give it a run with the differenciated series / the log of the data.

There is still the question how to test for significance of the found seasonality.

 

Stephan

 

 


Von: Charles Warner [mailto:[hidden email]]
Gesendet: Freitag, 18. November 2011 00:34
An: [hidden email]
Betreff: Re: [scilab-Users] saisonality in time series

 

Although "seasonality" is not the term I use for long term trends hidden in noisy data, I have had some success by taking the log of the data, and running an FFT on the log data.  Usually, I have some prior knowledge of the long-term periodic trends I expect, so it is relatively easy to determine quickly if this method works.  Plotting the log of the data also gives one a good feel for whether the data is stationary, or whether there are windows of data that can be treated as stationary.  Any changing magnitude effect is, of course, reduced when on works with logs, but such effects can help one understand what the raw data is really telling you.

Charlie

On Thu, Nov 17, 2011 at 12:40 PM, Mike Page <[hidden email]> wrote:

Hi,

I don't know much about this application, but the Cepstrum can be used to
find hidden periodicity in time series.  Might be worth trying?  I have used
it for finding rotational components in the vibration signatures from
rotating machinery.  There's a simple example here
(http://www.dliengineering.com/downloads/cepstrum%20analysis.pdf).

Mike.



-----Original Message-----
From: Petter Wingren [mailto:[hidden email]]
Sent: 17 November 2011 17:18
To: [hidden email]
Subject: Re: [scilab-Users] saisonality in time series


Did a quick search but couldnt find anything obvious. I suppose the
word you are looking for is seasonality - maybe that helps in finding
something useful.

On Thu, Nov 17, 2011 at 3:36 PM, Schreckenbach Stephan
<[hidden email]> wrote:
>
> Hi,
>
> I look for a test of saisonality in time series.
> The time series might be instationary and nonlinear and the saisonality
> / oscillation might have a changing amplitude. Furthermore the
> distribution
> might be unknown as well.
> I need something to test for significant saisonality without knowing /
> estimating a (linear) model of the time series.
>
> ideas I got so far: Chi Square Test for independency:
> I could test for independence of saison and mean value of the data
>
> Chi Square Test to test for different means of two data groups.
> I could test for a difference of the mean between several seasons.
>
> Any more or better ideas?
>
> Thanks in advance, Stephan
>
>

 

 




Summary.odt (399K) Download Attachment
Loading...