Readers,
There is no digest mode for this mailing list. Is it possible to subscribe but receive no mail please? 
On 17/11/2011, eletter <[hidden email]> wrote:
> Readers, > > There is no digest mode for this mailing list. Is it possible to > subscribe but receive no mail please? > Sorry, please ignore. 
In reply to this post by eletter
Le jeudi 17 novembre 2011 à 08:57 +0000, eletter a écrit :
> Readers, > > There is no digest mode for this mailing list. Is it possible to > subscribe but receive no mail please? For those who are not familiar, the mailing list are also available here: http://mailinglists.scilab.org/ And can be used like a forum if you prefer... Sylvestre 
Schreckenbach Stephan 
Hi, I look for a test of saisonality in time series. The time series might be instationary and nonlinear and the saisonality / oscillation might have a changing amplitude. Furthermore the distribution might be unknown as well. I need something to test for significant saisonality without knowing / estimating a (linear) model of the time series. ideas I got so far: Chi Square Test for independency: I could test for independence of saison and mean value of the data Chi Square Test to test for different means of two data groups. I could test for a difference of the mean between several seasons. Any more or better ideas? Thanks in advance, Stephan 
Adrien VogtSchilb 
I don't knwo if that is usefull for you:
I had to test seasonality and i did it successfully with Principal component analyses you may use the grocer atoms toolbox to do PCAs there's a lot of documentation on the web: https://www.google.com/search?q=seasonality+principal+component+analyses&hl=en On 17/11/2011 14:07, Schreckenbach Stephan wrote: Hi, I look for a test of saisonality in time series. The time series might be instationary and nonlinear and the saisonality / oscillation might have a changing amplitude. Furthermore the distribution might be unknown as well. I need something to test for significant saisonality without knowing / estimating a (linear) model of the time series. ideas I got so far: Chi Square Test for independency: I could test for independence of saison and mean value of the data Chi Square Test to test for different means of two data groups. I could test for a difference of the mean between several seasons. Any more or better ideas? Thanks in advance, Stephan 
Adrien VogtSchilb (Cired) Tel: (+33) 1 43 94 73 77 
Schreckenbach Stephan 
In reply to this post by Schreckenbach Stephan
Hi, I look for a test of saisonality in time series. The time series might be instationary and nonlinear and the saisonality / oscillation might have a changing amplitude. Furthermore the distribution might be unknown as well. I need something to test for significant saisonality without knowing / estimating a (linear) model of the time series. ideas I got so far: Chi Square Test for independency: I could test for independence of saison and mean value of the data Chi Square Test to test for different means of two data groups. I could test for a difference of the mean between several seasons. Any more or better ideas? Thanks in advance, Stephan 
Petter Wingren 
Did a quick search but couldnt find anything obvious. I suppose the
word you are looking for is seasonality  maybe that helps in finding something useful. On Thu, Nov 17, 2011 at 3:36 PM, Schreckenbach Stephan <[hidden email]> wrote: > > Hi, > > I look for a test of saisonality in time series. > The time series might be instationary and nonlinear and the saisonality > / oscillation might have a changing amplitude. Furthermore the > distribution > might be unknown as well. > I need something to test for significant saisonality without knowing / > estimating a (linear) model of the time series. > > ideas I got so far: Chi Square Test for independency: > I could test for independence of saison and mean value of the data > > Chi Square Test to test for different means of two data groups. > I could test for a difference of the mean between several seasons. > > Any more or better ideas? > > Thanks in advance, Stephan > > 
Hi,
I don't know much about this application, but the Cepstrum can be used to find hidden periodicity in time series. Might be worth trying? I have used it for finding rotational components in the vibration signatures from rotating machinery. There's a simple example here (http://www.dliengineering.com/downloads/cepstrum%20analysis.pdf). Mike. Original Message From: Petter Wingren [mailto:[hidden email]] Sent: 17 November 2011 17:18 To: [hidden email] Subject: Re: [scilabUsers] saisonality in time series Did a quick search but couldnt find anything obvious. I suppose the word you are looking for is seasonality  maybe that helps in finding something useful. On Thu, Nov 17, 2011 at 3:36 PM, Schreckenbach Stephan <[hidden email]> wrote: > > Hi, > > I look for a test of saisonality in time series. > The time series might be instationary and nonlinear and the saisonality > / oscillation might have a changing amplitude. Furthermore the > distribution > might be unknown as well. > I need something to test for significant saisonality without knowing / > estimating a (linear) model of the time series. > > ideas I got so far: Chi Square Test for independency: > I could test for independence of saison and mean value of the data > > Chi Square Test to test for different means of two data groups. > I could test for a difference of the mean between several seasons. > > Any more or better ideas? > > Thanks in advance, Stephan > > 
Charles Warner 
Although "seasonality" is not the term I use for long term trends hidden in noisy data, I have had some success by taking the log of the data, and running an FFT on the log data. Usually, I have some prior knowledge of the longterm periodic trends I expect, so it is relatively easy to determine quickly if this method works. Plotting the log of the data also gives one a good feel for whether the data is stationary, or whether there are windows of data that can be treated as stationary. Any changing magnitude effect is, of course, reduced when on works with logs, but such effects can help one understand what the raw data is really telling you.
Charlie On Thu, Nov 17, 2011 at 12:40 PM, Mike Page <[hidden email]> wrote: Hi, 
Schreckenbach Stephan 
Hi, sorry, of course I meant
seasonality. The time series consists
of longer term trends, short term noise and short time seasonality. oscillations /
seasonality, if any, it is most likely to be nonharmonic. I look for distinct
frequencies. When I did a FFT plot of
the original time series there was noise only in the spectrum. I will give it a run with
the differenciated series / the log of the data. There is still the
question how to test for significance of the found seasonality. Stephan Von: Charles Warner
[mailto:[hidden email]] Although
"seasonality" is not the term I use for long term trends hidden in
noisy data, I have had some success by taking the log of the data, and running
an FFT on the log data. Usually, I have some prior knowledge of the
longterm periodic trends I expect, so it is relatively easy to determine
quickly if this method works. Plotting the log of the data also gives one
a good feel for whether the data is stationary, or whether there are windows of
data that can be treated as stationary. Any changing magnitude effect is,
of course, reduced when on works with logs, but such effects can help one
understand what the raw data is really telling you. On Thu, Nov 17, 2011 at 12:40 PM, Mike Page <[hidden email]>
wrote: Hi,

Charles Warner 
Another trick I have found that greatly reduces FFT noise it to temporarily mask any localized "spikes" in the data (such spikes, with a narrow temporal profile have a very broad spectral distribution). One can also try to eliminate any offset by subtracting the mean (or the geometric mean or harmonic mean the appropriate mean would be dictated by the nature of the data). This should hopefully reduce the scale of the FFT amplitude, making it easier to spot any (especially lowfrequency, or seasonal) potential frequency components.
On Fri, Nov 18, 2011 at 3:09 AM, Schreckenbach Stephan <[hidden email]> wrote:

Schreckenbach Stephan 
Filtering temporal spikes
is a good idea, since there are some of them. I will try that. The data sample as around
7000 data points, the frequency I look for is around 1/10 * sample rate. May be there are methods
that are better suited for identifying frequency components in that kind of
data? FFT always describes the
time series by harmonic oszillations, which might not work well if oscillations are not (strictly)
harmonic. What about wavelets (don’t
know much about it yet, though)? Stephan Von: Charles Warner
[mailto:[hidden email]] Another trick I have
found that greatly reduces FFT noise it to temporarily mask any localized
"spikes" in the data (such spikes, with a narrow temporal profile
have a very broad spectral distribution). One can also try to eliminate
any offset by subtracting the mean (or the geometric mean or harmonic mean the
appropriate mean would be dictated by the nature of the data). This
should hopefully reduce the scale of the FFT amplitude, making it easier to
spot any (especially lowfrequency, or seasonal) potential frequency
components. On Fri, Nov 18, 2011 at 3:09 AM, Hi, sorry, of course I meant seasonality. The time series consists of longer term trends,
short term noise and short time seasonality. oscillations / seasonality, if any, it is most
likely to be nonharmonic. I look for distinct frequencies. When I did a FFT plot of the original time series
there was noise only in the spectrum. I will give it a run with the differenciated
series / the log of the data. There is still the question how to test for
significance of the found seasonality. Stephan Von: Charles Warner [mailto:[hidden email]]
Although
"seasonality" is not the term I use for long term trends hidden in
noisy data, I have had some success by taking the log of the data, and running
an FFT on the log data. Usually, I have some prior knowledge of the
longterm periodic trends I expect, so it is relatively easy to determine
quickly if this method works. Plotting the log of the data also gives one
a good feel for whether the data is stationary, or whether there are windows of
data that can be treated as stationary. Any changing magnitude effect is,
of course, reduced when on works with logs, but such effects can help one
understand what the raw data is really telling you. On Thu, Nov 17,
2011 at 12:40 PM, Mike Page <[hidden email]> wrote: Hi,

In reply to this post by Charles Warner
Better stick with DFT, smoothed DFT or try seasonal adjustment freeware Demetra+  that's what official statisticians might do.
gin On Mon, Nov 21, 2011 at 10:00 AM, Schreckenbach Stephan <[hidden email]> wrote:

Charles Warner 
Stephan
Sounds like you are working with data sets that resemble some that I work with frequently. About the same size, nonstationary seasonality (which I prefer to call lowfrequency periodic trends), high, nonstationary temporally variable "noisy" signals. I usually have a priori knowledge that there is likely aliasing due to the fact that I am limited in sampling rate vs. total time period (varying the sampling rate for different collection instances can sometimes resolve at least part of this issue). I haven't figured out how to get the aliasing out of the system, although there should be some way to do this based on "unfolding" the spectrum. Anyway, the approach I take is this. Starting with a COPY of the raw data, I run it through NIST Dataplot to get a feel for the data, using what they call the 4plot. This gives me a feel for the periodicity, autocorrelation, statistical distribution, and helps me identify the "ouliers" (i.e., spikes, 0's, any data point that is "unusual"), which are then filtered out by replacing specific data points with a "more reasonable" value (which is why I use a copy of the raw data, rather than the original). Usually use a spreadsheet for this. I might run the data through Dataplot a couple of times to evaluate the effects my "filtering" have had. Usually, still in the spreadsheet, I next remove the "DC offset" by subtracting one of the Pythagorean means. Which is appropriate depends on the nature of the data. This reduces the low end of the spectrum, which is where the "trends" are located. For the Fourier analysis, I could stay with Dataplot, but I find it much easier to extract information from the Scilab approach. Furthermore, Scilab offers an alternative DFT processcalled MESE. The Maximum Entropy Spectral Estimate (MESE), designed to produce highresolution, lowbias spectral estimate (refer to page 128 of the Signal processing With Scilab manual, or available here). MESE incorporates no information in the estimated spectrum about the autocorrelation lags. That is to say that the bias resulting from the leakage from the window sidelobes should be eliminated (or at least minimized in some sense). In other words, one tends to get cleaner "spikes" in the spectrum. It is much easier to pick out the lowerfrequency components of the signals with this procedure. One then subtracts these components from the working data, and repeats the process. Ideally, you have extracted all of the available information from the data when the residual is Gaussian white noise (a point I have never actually reached in practice). I don't believe "windowing" techniques will work with lowfrequency components, although I could be mistaken in this. I have toyed with windowing when I have reduced my residual to what has the appearance of a frequencymodulated signal but, then, I am looking to characterize such events. The information you are trying to extract will ultimately dictate what approach you take. I am attaching a "working document" that I have put together giving more detail on this approach. Charlie 2011/11/21 Ginters Bušs <[hidden email]> Better stick with DFT, smoothed DFT or try seasonal adjustment freeware Demetra+  that's what official statisticians might do. Summary.odt (399K) Download Attachment 
Free forum by Nabble  Edit this page 