Scrolling very large datasets

Need help in implementing some specific function to your LightningChart Ultimate powered application? Post a question and get code snippets from other LightningChart Ultimate community members.

Moderator: Queue Moderators

Post Reply
fischertc
Posts: 18
Joined: Wed Feb 12, 2014 6:16 am

Scrolling very large datasets

Post by fischertc » Thu Apr 16, 2015 10:33 am

I am using WPF and 6.4.3 and point line series.

I have some very large data files that cannot fit into memory (many GB) so I have implemented scrolling myself. I am up for suggestions on the fastest way to scroll when you are adding data to the beginning of the points array.

The standard chart view size is 100 meters which is about 200,000 rows of data. I have 19 to 21 Y axes/PointLineSeries depending on the file.

Adding to the end of the points array when scrolling right is fairly straight forward, delete anything prior to what you want and add the small amount of points to the end.

Code: Select all

_chart.ViewXY.PointLineSeries[i].DeletePointsBeforeX(startPos);
_chart.ViewXY.PointLineSeries[i].AddPoints(locations, channelData, false);
But going in the other direction is more problematic and much slower:

Code: Select all


// Create an array of points for the new data
SeriesPoint[] frontPoints = new SeriesPoint[addToBegin];

// Move the read in data to the new points array
Parallel.For(0, addToBegin, k => { frontPoints[k].X = locations[k]; frontPoints[k].Y = channelData[k]; });

// Concat the new points and old points together
// Set the complete points array in the series
_chart.ViewXY.PointLineSeries[i].Points = frontPoints.Concat(_chart.ViewXY.PointLineSeries[i].Points.Take(_chart.ViewXY.PointLineSeries[i].PointCount - addToBegin)).ToArray();;

Scrolling right is pretty smooth but scrolling left has lots of hesitation. Has anyone got any ideas on how to make this faster?

User avatar
ArctionPasi
Posts: 1367
Joined: Tue Mar 26, 2013 10:57 pm
Location: Finland
Contact:

Re: Scrolling very large datasets

Post by ArctionPasi » Thu Apr 16, 2015 2:48 pm

You probably are also using ScrollPosition setting in real-time monitoring? LC's series support progressing AddPoints/Addsamples and ScrollPosition by X, and is very efficient. Backwards it is not implemented the same way.

If you want to scroll data efficiently from right to left, you could try the following approach:
- set xAxis.Reversed = true;
- feed the data in reversed order (flip the AddPoints or AddSamples contents). Ensure x[i+1] >= x.
- set ScrollPosition to last added point X
- Manipulate X axis labels with xAxis.FormatValueLabel event handler.

Might be a little bit tricky but others have done it in similar way.

That way scrolling from right to left has the same performance than from left to right.
LightningChart Support Team, PT

fischertc
Posts: 18
Joined: Wed Feb 12, 2014 6:16 am

Re: Scrolling very large datasets

Post by fischertc » Fri Apr 17, 2015 3:12 am

I cannot get this to work.

Reversing the X axis and calling AddPoints() with low X value, reversed data just added the reversed data to the high X value end of the points array which is what is does when the axis is not reversed.

If you could post a bit of code, I am sure I am missing something here.

User avatar
ArctionPasi
Posts: 1367
Joined: Tue Mar 26, 2013 10:57 pm
Location: Finland
Contact:

Re: Scrolling very large datasets

Post by ArctionPasi » Fri Apr 17, 2015 10:37 am

Ok, I'll try to make an example application in the beginning of next week.
LightningChart Support Team, PT

User avatar
ArctionPasi
Posts: 1367
Joined: Tue Mar 26, 2013 10:57 pm
Location: Finland
Contact:

Re: Scrolling very large datasets

Post by ArctionPasi » Tue Apr 21, 2015 1:14 pm

I'm delayed because of new assembly pack released today. I'll return to this matter asap.
LightningChart Support Team, PT

User avatar
ArctionPasi
Posts: 1367
Joined: Tue Mar 26, 2013 10:57 pm
Location: Finland
Contact:

Re: Scrolling very large datasets

Post by ArctionPasi » Wed Apr 22, 2015 3:53 pm

I'm sorry for the delay :oops:
WpfDataScrollingBackwardsVS2010.zip
VS2010 C# WPF project
(10.06 KiB) Downloaded 714 times
I used a spikes data to clearly point out the X and Y values relation and movement of spikes as they scroll left or right. Generate your own data and put as many points as you want... It scrolls similarly good in both directions.
Backwards scrolling example
Backwards scrolling example
Scrolling backwards.jpg (437.21 KiB) Viewed 14772 times
I hope this helps :)
LightningChart Support Team, PT

fischertc
Posts: 18
Joined: Wed Feb 12, 2014 6:16 am

Re: Scrolling very large datasets

Post by fischertc » Thu Apr 23, 2015 4:34 am

Thank you!

Post Reply