How to plot a smooth real time data from list<> of data

Need help in implementing some specific function to your LightningChart Ultimate powered application? Post a question and get code snippets from other LightningChart Ultimate community members.

Moderator: Queue Moderators

Post Reply
jhyap
Posts: 28
Joined: Tue Apr 16, 2013 12:07 am

How to plot a smooth real time data from list<> of data

Post by jhyap » Tue May 07, 2013 12:52 pm

Hi Arction Team,

I received my data from the TCP server. Every 1 second, I will receive 100 data. The data is packed inside a list<>. The list contain two parameter: amplitude and time. As shown below:
List size is 100, contain amplitude and DateTime information
List size is 100, contain amplitude and DateTime information
List of data.JPG (50.13 KiB) Viewed 57558 times
I pack the list into SeriesPoint[] using a simple for loop. As shown below:

Code: Select all

                        myList = myData.ToList<ChannelValue>();

                        SeriesPoint[] aPoints = new SeriesPoint[myList.Count];

                        for (int i = 0; i < myList.Count; i++)
                        {
                            m_dLatestX = m_chart.ViewXY.XAxes[0].DateTimeToAxisValue(myList[i].Time);
                            aPoints[i].X = m_dLatestX;
                            aPoints[i].Y = myList[i].Data;
                        }
                        PlotSeries(aPoints);
Then, I pass the aPoints into method: PlotSeries(aPoints). As shown below:

Code: Select all

        private void PlotSeries(SeriesPoint[] aPoints)
        {
            m_chart.BeginUpdate();

            m_chart.ViewXY.YAxes[0].SetRange(0, 100);

            PointLineSeries pls = new PointLineSeries(m_chart.ViewXY, m_chart.ViewXY.XAxes[0], m_chart.ViewXY.YAxes[0]);
            pls.PointsVisible = false;

            pls.Points = aPoints;
            m_chart.ViewXY.PointLineSeries.Add(pls);

            m_chart.ViewXY.XAxes[0].ScrollPosition = m_dLatestX;

            m_chart.EndUpdate();
        }
And below is my result:
Result of real time graph using PointLineSeries
Result of real time graph using PointLineSeries
Result.JPG (49.35 KiB) Viewed 57558 times
Event though it looks visually fine in the picture. But this method is plotting all the 100 data per 1 second (since I am receiving 100 data per 1 second using timer). It does not looks smooth for real time. This is because, the graph will instantly print all the 100 data on graph every 1 second. Any good idea to plot data real time from a list<>?

Regards,
:firing: JH

ArctionJari

Re: How to plot a smooth real time data from list<> of data

Post by ArctionJari » Tue May 07, 2013 2:57 pm

Hi JH.

You could create a data buffer that stores the samples you receive from your TCP server in the background. Then you use a thread to fetch all the data that is available in that buffer - let's say every 100th millisecond. You need to adjust the thread sleep (i.e. data polling frequency) yourself to make the data scroll look smooth but at the same time keeping GUI updates to a minimum. For example, if you receive 100 samples per second from your TCP server to your buffer and you poll samples from your buffer every 10th millisecond you only get one sample for rendering. Rendering one sample at a time isn't a very good practice because calling chart's rendering routines etc takes more time than rendering that one sample. On the other hand, your buffer gets filled in the background with more than one samples when you are rendering so it depends on how you implement your whole data buffer / rendering system. :freak:

In a nut shell, there's no point in polling your buffer frequently if there are no samples available. :)

Let me know if this is the thing you needed to solve your problem.

jhyap
Posts: 28
Joined: Tue Apr 16, 2013 12:07 am

Re: How to plot a smooth real time data from list<> of data

Post by jhyap » Tue May 07, 2013 3:00 pm

Ok. I grab some picture. Will try it tomorrow. Thanks :D

jhyap
Posts: 28
Joined: Tue Apr 16, 2013 12:07 am

Re: How to plot a smooth real time data from list<> of data

Post by jhyap » Mon May 13, 2013 2:08 pm

Hi Jari,

I perform my real time multi-channel graph as the following. The chart does not seems to be pretty because I am using timer to fire a whole bunch of multi-channel data every 1 second. I am having problem on how to use thread to thread multi-channel data on chart. May be you can have some comment on the following method and point some opinion.

I am using PointLineSeries , SeriesPoint[][] to plot my mulit-channel data on chart.

Firstly, I have a timer, to get data table from a buffer. Inside the data table, it contains 9 channels data, and each channels consist of data size 200~400 (it will be variable length, and the length varies on every timer fired.) And each data, contain two information: Data and Time (As shown in my previous picture in this post)

My 3rd party timer in Main(), the timer is ticked every 1 second:

Code: Select all

            m_timerThread = new Multimedia.Timer();
            m_timerThread.Mode = TimerMode.Periodic;
            m_timerThread.Resolution = 1;
            m_timerThread.Period = 1000;
            m_timerThread.Tick += new EventHandler(m_timerThread_Tick);
            m_timerThread.Start(); 
I get data table from a buffer per timer ticked:

Code: Select all

void m_timerThread_Tick(object sender, EventArgs e)
{
                //receiver.GetTable() will get the a Data Table from buffer 
                lock (receiver.GetTable())
                {
                    // receiver.GetTable().Count will give the 
                    // information of total channel inside the Data Table
                    m_iChannelCount = (int)receiver.GetTable().Count;

                    // channel counter to track the number of channel 
                   // inside the foreach loop
                    m_iChannel = 0;
                    
                   // I use SeriesPoint[][] to create a 2D array to fit multi-channel data
                    SeriesPoint[][] aMultiChannelData = new SeriesPoint[m_iChannelCount][];
 
                    // This foreach loop is looped to get data for each of 
                    // the channel inside the Data Table
                    foreach (var c in receiver.GetTable())
                    {
                        var channel = c as ChannelInfo; 
                        var data = channel.GetData();

                        myList = data.ToList<ChannelValue>();
                        
                        // Define the data size for each channel
                        aMultiChannelData[m_iChannel] = new SeriesPoint[data.Length];

                        // Loop on single channel to copy data from buffer 
                        // into the aMultiChannelData[m_iChannel][i]
                        for (int i = 0; i < myList.Count; i++)
                        {
                            aMultiChannelData[m_iChannel][i].Y = myList[i].Data;
                            m_dLatestX = m_chart.ViewXY.XAxes[0].DateTimeToAxisValue(myList[i].Time);
                            aMultiChannelData[m_iChannel][i].X = m_dLatestX;
                        }
                       // After finish copying data from buffer into  
                       // aMultiChannelData[m_iChannel][i], channel counter +1
                        m_iChannel++;
                    }
                    m_iChannel = 0; // Reset the channel counter to 0
 
                    // I feed all the data into this method and update the chart
                    FeedMultiDataToChart(aMultiChannelData, m_iChannelCount);
                    Thread.Sleep(10);
                }
}
Here, I update the chart and render all the data inside SeriesPoint[][]

Code: Select all

private void FeedMultiDataToChart(SeriesPoint[][] data, int iChannelCount)
{
                m_chart.BeginUpdate();
                for (int iChannel = 0; iChannel < iChannelCount; iChannel++)
                {
                    m_chart.ViewXY.PointLineSeries[iChannel].AddPoints(data[iChannel], false);
                }
                m_dLatestX = m_chart.ViewXY.XAxes[0].DateTimeToAxisValue(DateTime.Now);
                m_chart.ViewXY.XAxes[0].ScrollPosition = m_dLatestX;
                m_chart.EndUpdate();
}
Picture to show a 1 tick data array: (Note that, not all the data size are the same among the channels, some channel has 500 data size, some channel has 400 data size, and some has data size of 200)
DataArray using SeriesPoint[][] for 1 tick
DataArray using SeriesPoint[][] for 1 tick
FeedDataArrayIntoMethod.JPG (59.93 KiB) Viewed 57520 times
So, my question is.. how to implement threading to thread multi channel data into chart? Or is there any better idea or tool to render real time chart using lightingChart?

jhyap
Posts: 28
Joined: Tue Apr 16, 2013 12:07 am

Re: How to plot a smooth real time data from list<> of data

Post by jhyap » Tue May 14, 2013 7:31 am

Today, I replaced the timer with a thread. And it seems to have minor improvement.

Result if use Timer:
Result of real time mulitchannel data rendering using Timer
Result of real time mulitchannel data rendering using Timer
TimerResult.JPG (204.95 KiB) Viewed 57513 times
Result if use Thread:
Result of multichannel data rendering using Thread
Result of multichannel data rendering using Thread
ThreadResult.JPG (191.7 KiB) Viewed 57513 times
Any opinion? :freak:

ArctionJari

Re: How to plot a smooth real time data from list<> of data

Post by ArctionJari » Tue May 14, 2013 11:06 am

Hi JH.

First of all, I would recommend that you do not use timers. Is that MultiMedia timer your own implementation? You should also know that there are timers that run synchronously with the application's main thread and timers that operate in their own separate thread. If you try to update your chart using non-synchronized timer/thread you need to call Control.Invoke (WinForms) or DispatcherObject.Invoke (WPF) to synchronize GUI updates. Otherwise you will encounter problems (crashing, freezing, ...).

I'm not exactly sure what is the problem you are having... Have you looked into LightningChart Ultimate demo application's "Thread-fed multi-channel data" example under "Real-time monitoring"? This example demonstrates how to use a thread to add points to your chart in real-time. It uses SampleDataSeries instead of PointLineSeries but the idea is the same. If possible use SampleDataSeries in your application, too. If you use SampleDataSeries you don't have to calculate X values. In initialization you just provide sampling frequency and the first sample timestamp and SampleDataSeries handles the rest when you call SampleDataSeries.AddSamples method. If your data is not sub-sequent then you have to use PointLineSeries - i.e. the "distance" between two data points is not always constant.

The idea is that your data buffer, that receives samples from ethernet port, has some constant length - for example 5 seconds. You then use thread to get all available samples from the buffer, for example every 10th millisecond, and update your chart with the new samples. If you do not fetch any samples from your buffer within that 5 seconds, it overflows and stops collecting data from ethernet port. A good idea is to provide an error code or something similar the next time you try to fetch samples from the overflown buffer so that your application can tell the user that samples were not collected fast enough.

Your data buffer has to be separate from your main application. In other words, the main application does not wait for the data buffer or vice versa. You just fetch samples from it until you stop the measurement session.

I hope this helps you on your quest for lost samples. :mrgreen:

jhyap
Posts: 28
Joined: Tue Apr 16, 2013 12:07 am

Re: How to plot a smooth real time data from list<> of data

Post by jhyap » Wed May 22, 2013 1:16 pm

Hi Jari,

I am still using PointLineSeries. And I am having these problem now:

The timing between data receive from buffer and feed to chart seems to be crucial

When there is no data inside the buffer, the data chart will be dis-join. Then the last point of data will go GG

And when new data pull from buffer, the 1st point of new data will go GG as well

Any idea on this..? :freak:
Continous add the endpoint even no data.JPG
Continous add the endpoint even no data.JPG (113.78 KiB) Viewed 57401 times

ArctionJari

Re: How to plot a smooth real time data from list<> of data

Post by ArctionJari » Wed May 22, 2013 2:06 pm

Hi JH.

You could use another timer to divide the data handling into smaller pieces. In other words, get a small piece of data (~ 10 samples) from the buffer for example 10 times in a second. You could use an event to provide data to the chart. This way you can make the data scrolling smoother. Now that I know more about this problem my previous idea was a little bit complicated... :roll:

For the sake of clarity we will provide you with an example Visual Studio project that demonstrates how this is done. It's sometimes hard to describe things just by words. :D

We will reply to this topic in a day or two with VS project and some code snippets.

jhyap
Posts: 28
Joined: Tue Apr 16, 2013 12:07 am

Re: How to plot a smooth real time data from list<> of data

Post by jhyap » Wed May 22, 2013 3:49 pm

thanks jari. tomorrow i will try to use timer inside the thread to segment the huge date set into smaller piece, while waiting your reply :)

new problems just keep on poping out when comes to real time implementation. :freak:

jhyap
Posts: 28
Joined: Tue Apr 16, 2013 12:07 am

Re: How to plot a smooth real time data from list<> of data

Post by jhyap » Thu May 23, 2013 8:56 am

Considering my number of data might be too large to render at a time. Hence, I decided to fixed the data pull from buffer to be 100 only. and update 100 data only into chart at one time

logic:

--[wait 100ms second to get 100 data from buffer] -- [feed 100 data to chart] --[wait 1ms to wait for chart update ]--[feed 100 data to chart] --->

With lower data count at a time, the graph is slower, whereas the last point of old data and first point of new data does not go GG. But, it appears the problem below:
timing between data receive from buffer and updating the chart is crucial
timing between data receive from buffer and updating the chart is crucial
timing between data receive from buffer and updating the chart is crucial.JPG (25.6 KiB) Viewed 57391 times

my data array to feed into chart:

[0][100]
[1][100]
[2][100]
.
.
.
[39][100]

User avatar
ArctionPasi
Posts: 1367
Joined: Tue Mar 26, 2013 10:57 pm
Location: Finland
Contact:

Re: How to plot a smooth real time data from list<> of data

Post by ArctionPasi » Thu May 23, 2013 7:54 pm

Hi jhyap,

I wrote a simple example, where
- For simplicity, only 1 series. The principle is the same for several series.
- Timer generates 100 new data points each second, and stores into a buffer.
- Another timer reads data from the buffer in approx 50 ms intervals, giving 20 refreshes / sec.
So the 50 timer distributes the updates in small pieces and forwards to the chart.

Code: Select all

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using Arction.LightningChartUltimate;
using Arction.LightningChartUltimate.SeriesXY;


namespace DataSmooth
{
    public partial class Form1 : Form
    {
        LightningChartUltimate m_chart;
        List<SeriesPoint> m_dataBuffer = new List<SeriesPoint>();
        Random m_rand = new Random();
        const double IntervalX = 0.01;  //interval between data points. 1 sec / 100 points/s 
        int m_iDataIndex = 0;
        object m_bufferLock = new object();
        int m_iCountOfPointsSmoothRelayerMustDeliverEachRound = 0;


        public Form1()
        {
            InitializeComponent();

            CreateChart();

            timerData1000.Start(); //1 second timer
            timerSmoothRelayer.Start(); //50 ms timer
        }

        void CreateChart()
        {
            m_chart = new LightningChartUltimate("Y3PT77S2RQRCKU6M6PR2EQ7XKJEHMSG9WHAZUQAHZLWY7RTFYU74CP6UT9DGB7AFJVTJB35PML8RWY4HBGAVHL9NP9WEGRVU2R9FF5JXWVQZMJ9T4TZMLYK65EX9WCSSVSFRTZKVUJCVKHJPJFVYXD75XVDUMJSFK2ZKJHLLRGS9J9B4M6W2");
            m_chart.Parent = this;
            m_chart.Dock = DockStyle.Fill;
            m_chart.BeginUpdate();

            PointLineSeries series1 = new PointLineSeries(m_chart.ViewXY, m_chart.ViewXY.XAxes[0], m_chart.ViewXY.YAxes[0]);
            m_chart.ViewXY.PointLineSeries.Add(series1);

            //Enable automatic data destruction 
            m_chart.ViewXY.DropOldSeriesData = true;

            //Set scrolling mode on 
            m_chart.ViewXY.XAxes[0].ScrollMode = XAxisScrollMode.Scrolling;

            //Use time scale, 0 in the measurement start 
            m_chart.ViewXY.XAxes[0].ValueType = AxisValueType.Time;


            m_chart.EndUpdate();

        }

        private void timerData1000_Tick(object sender, EventArgs e)
        {
            //Generate 100 data points, every 1000 ms 
            int iPointCount = 100;

            //List <SeriesPoint> newDataPoints = new List<SeriesPoint>(iPointCount); 
            SeriesPoint[] newDataPoints = new SeriesPoint[iPointCount];

            for (int i = 0; i < iPointCount; i++)
            {
                newDataPoints[i].X = (double)m_iDataIndex * IntervalX;
                newDataPoints[i].Y = m_rand.NextDouble() * 10.0;
                m_iDataIndex++;
            }

            //Lock is needed when using a Thread or a timer that doesn't have Invoke in UI as built-in. 
            //In this example, System.Windows.Forms.Timer it is NOT needed but left here to avoid confusion with others. 
            lock (m_bufferLock)
            {
                m_dataBuffer.AddRange(newDataPoints);

                //Set how many points the smooth relayer should deliver each round.
                //Automatically grows the delivered size if the software can't keep in pace otherwise. 
                m_iCountOfPointsSmoothRelayerMustDeliverEachRound = m_dataBuffer.Count / timerSmoothRelayer.Interval;
            }
        }

        private void timerSmoothRelayer_Tick(object sender, EventArgs e)
        {
            lock (m_bufferLock)
            {
                int iTakeCount = m_iCountOfPointsSmoothRelayerMustDeliverEachRound;
                if (m_dataBuffer.Count < iTakeCount)
                    iTakeCount = m_dataBuffer.Count;

                //Create array of points to be relayed to the chart 
                SeriesPoint[] relayPoints = new SeriesPoint[iTakeCount];

                for (int i = 0; i < iTakeCount; i++)
                {
                    relayPoints[i] = m_dataBuffer[i];
                }

                //Remove points from buffer 
                m_dataBuffer.RemoveRange(0, iTakeCount);

                FeedDataToChart(relayPoints);

            }
        }

        void FeedDataToChart(SeriesPoint[] points)
        {
            if (points == null || points.Length == 0)
                return;

            m_chart.BeginUpdate();

            //Add points to the series
            m_chart.ViewXY.PointLineSeries[0].AddPoints(points, false);

            //Update real-time scroll position
            m_chart.ViewXY.XAxes[0].ScrollPosition = points[points.Length - 1].X;

            m_chart.EndUpdate();
        }
    }
}


Like this the data flows smoothly, instead of 1 refresh / sec.

8-)
LightningChart Support Team, PT

jhyap
Posts: 28
Joined: Tue Apr 16, 2013 12:07 am

Re: How to plot a smooth real time data from list<> of data

Post by jhyap » Fri May 24, 2013 9:07 am

Impressive smooth :o

So does this mean, for each of the channel/series, I need to create 1 buffer for fast rendering?

:freak:

User avatar
ArctionPasi
Posts: 1367
Joined: Tue Mar 26, 2013 10:57 pm
Location: Finland
Contact:

Re: How to plot a smooth real time data from list<> of data

Post by ArctionPasi » Fri May 24, 2013 9:21 am

If all your channels use same data point X interval, I'd suggest using only one buffer and including an array for every channel there.

So, instead of
List<SeriesPoint> m_dataBuffer = new List<SeriesPoint>();

use it like
List<SeriesPoint[]> m_dataBuffer = new List<SeriesPoint[]>();

:shock:
LightningChart Support Team, PT

jhyap
Posts: 28
Joined: Tue Apr 16, 2013 12:07 am

Re: How to plot a smooth real time data from list<> of data

Post by jhyap » Fri May 24, 2013 9:33 am

Thanks :D

NadaNaeem
Posts: 1
Joined: Mon Jul 01, 2013 8:38 am

Re: How to plot a smooth real time data from list<> of data

Post by NadaNaeem » Mon Jul 01, 2013 9:38 am

Hello jhyap,
I am working on a similar project,
I kindly would like to ask you if
it is possible to give me your source code,
you will save me a lot of time,

Regards,
Nada.

jhyap
Posts: 28
Joined: Tue Apr 16, 2013 12:07 am

Re: How to plot a smooth real time data from list<> of data

Post by jhyap » Mon Jul 01, 2013 9:53 am

Code: Select all

        // Latest value of series x, used to set x-axis scrolling position properly
        private double m_dLatestX = 0;

        // Total channel number
        private int m_iChannelCount;

        // Interval counter
        private int counter = 0;

        // TCP receive data from server
        ReceivePacket _receiver;

        // 2D array to store chart data
        SeriesPoint[][] aMultiChannelData;

        // data buffer to chart
        public List<SeriesPoint[]> m_dataBuffer = new List<SeriesPoint[]>();
        object m_bufferLock = new object();
        List<int> m_iCountOfPointsSmoothRelayerMustDeliverEachRound = new List<int>();

Code: Select all

private void timerData1000_Tick(object sender, EventArgs e)
        {
            if (!_receiver.IsRun)
            {
                _receiver.Start();
            }

            string[] mySelectedCode = get_selectedCode.ToArray();

            aMultiChannelData = new SeriesPoint[m_iChannelCount][];

            var plSeriesList = m_chart.ViewXY.PointLineSeries;

            for (int iChannel = 0; iChannel < m_iChannelCount; iChannel++)
            {
                string s = mySelectedCode[iChannel];

                if (!_receiver.GetTable().ContainsKey(s)) continue;

                var myDataTable = _receiver.GetTable()[s];

                if (!(myDataTable is ChannelInfo)) continue;

                ChannelInfo channel = myDataTable as ChannelInfo;

                var series = plSeriesList[iChannel];

                AxisX xAxis = m_chart.ViewXY.XAxes[0];

                if (series.PointCount == 0)
                {
                    myData = channel.GetData();
                }
                else
                {
                    DateTime lastpointTime = xAxis.AxisValueToDateTime(series.Points[series.PointCount - 1].X);
                    myData = channel.GetData(lastpointTime.AddMilliseconds(1));
                }

                if (myData == null || myData.Length == 0) continue;

                int myDataLen = (int)myData.Length;

                aMultiChannelData[iChannel] = new SeriesPoint[myData.Length];

                for (int iPoint = 0; iPoint < myDataLen; iPoint++)
                {
                    m_dLatestX = m_chart.ViewXY.XAxes[0].DateTimeToAxisValue(myData[iPoint].Time);
                    aMultiChannelData[iChannel][iPoint].X = m_dLatestX;
                    aMultiChannelData[iChannel][iPoint].Y = myData[iPoint].Data;
                }
            }
            lock (m_bufferLock)
            {
                m_dataBuffer.Clear();
                m_dataBuffer.AddRange(aMultiChannelData);

                foreach (SeriesPoint[] m_SeriesPointArray in m_dataBuffer)
                {
                    if (m_SeriesPointArray == null) continue;
                    m_iCountOfPointsSmoothRelayerMustDeliverEachRound.Add((int)m_SeriesPointArray.Length / timerSmoothRelayer.Interval);
                }
            }
        }

Code: Select all

private void timerSmoothRelayer_Tick_1(object sender, EventArgs e)
        {
            lock (m_bufferLock)
            {
                relayPoints = new SeriesPoint[m_dataBuffer.Count][];
                for (int index = 0; index < m_dataBuffer.Count; index++)
                {
                    if (m_dataBuffer[index] == null) continue;
                    int iTakeCount = m_iCountOfPointsSmoothRelayerMustDeliverEachRound[counter];
                    if (m_dataBuffer[index].Length < iTakeCount)
                        iTakeCount = m_dataBuffer[index].Length;

                    // Create array of points to be relayed to the chart
                    relayPoints[index] = new SeriesPoint[iTakeCount];

                    for (int i = 0; i < iTakeCount; i++)
                    {
                        relayPoints[index][i] = m_dataBuffer[index][i];
                    }
                    var tempList = m_dataBuffer[index].ToList();
                    tempList.RemoveRange(0, iTakeCount);
                    m_dataBuffer[index] = tempList.ToArray();
                    counter++;
                }
                
                FeedNewDataToChart(relayPoints);
                counter = 0;
            }
        }

Code: Select all

private void FeedNewDataToChart(SeriesPoint[][] data)
        {
            m_chart.BeginUpdate();
            ViewXY chartView = m_chart.ViewXY;
            for (int iChannel = 0; iChannel < data.Length; iChannel++)
            {
                if (data[iChannel] == null) continue;

                chartView.PointLineSeries[iChannel].AddPoints(data[iChannel], false);
                var series = chartView.PointLineSeries[iChannel];
                m_dLatestX = series.Points[series.PointCount - 1].X;

            }
            chartView.XAxes[0].ScrollPosition = m_dLatestX;
            m_chart.EndUpdate();
        }
Hope it helps :geek:

Post Reply