Class DataProcessor
- java.lang.Object
-
- org.rrd4j.data.DataProcessor
-
public class DataProcessor extends Object
Class which should be used for all calculations based on the data fetched from RRD files. This class supports ordinary DEF datasources (defined in RRD files), CDEF datasources (RPN expressions evaluation), SDEF (static datasources - extension of Rrd4j) and PDEF (plottables, see
Plottable
for more information.Typical class usage:
final long t1 = ... final long t2 = ... DataProcessor dp = new DataProcessor(t1, t2); // DEF datasource dp.addDatasource("x", "demo.rrd", "some_source", "AVERAGE"); // DEF datasource dp.addDatasource("y", "demo.rrd", "some_other_source", "AVERAGE"); // CDEF datasource, z = (x + y) / 2 dp.addDatasource("z", "x,y,+,2,/"); // ACTION! dp.processData(); // Dump calculated values System.out.println(dp.dump());
-
-
Field Summary
Fields Modifier and Type Field Description static double
DEFAULT_PERCENTILE
ConstantDEFAULT_PERCENTILE=95.0
static int
DEFAULT_PIXEL_COUNT
Constant representing the default number of pixels on a Rrd4j graph (will be used if no other value is specified withsetStep()
method.static boolean
DEFAULT_POOL_USAGE_POLICY
Constant that defines the defaultRrdDbPool
usage policy.
-
Constructor Summary
Constructors Constructor Description DataProcessor(long t1, long t2)
Creates new DataProcessor object for the given time span.DataProcessor(Calendar gc1, Calendar gc2)
Creates new DataProcessor object for the given time span.DataProcessor(Date d1, Date d2)
Creates new DataProcessor object for the given time span.
-
Method Summary
All Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description void
addDatasource(String name, String rpnExpression)
Adds complex source (CDEF).void
addDatasource(String name, String sourceName, double percentile)
Deprecated.UseVariable
based method instead.void
addDatasource(String name, String file, String dsName, ConsolFun consolFunc)
Adds simple datasource (DEF).void
addDatasource(String name, String file, String dsName, ConsolFun consolFunc, String backend)
Deprecated.void
addDatasource(String name, String file, String dsName, ConsolFun consolFunc, RrdBackendFactory backend)
Adds simple source (DEF).void
addDatasource(String name, String defName, ConsolFun consolFun)
Deprecated.UseVariable
based method instead.void
addDatasource(String name, String dsName, FetchData fetchData)
Adds DEF datasource with datasource values already available in the FetchData object.void
addDatasource(String name, String defName, Variable var)
Creates a datasource that performs a variable calculation on an another named datasource to yield a single combined timestampe/value.void
addDatasource(String name, FetchData fetchData)
Adds DEF datasource with datasource values already available in the FetchData object.void
addDatasource(String name, Plottable plottable)
Adds a custom,plottable
datasource (PDEF).String
dump()
Dumps timestamps and values of all datasources in a tabular form.double
get95Percentile(String sourceName)
Deprecated.UseVariable
based method instead.double
getAggregate(String sourceName, ConsolFun consolFun)
Deprecated.UseVariable
based method instead.Aggregates
getAggregates(String sourceName)
Deprecated.UseVariable
based method instead.long
getEndingTimestamp()
Returns ending timestamp.long
getFetchRequestResolution()
Returns desired RRD archive step (resolution) in seconds to be used while fetching data from RRD files.long
getLastRrdArchiveUpdateTime()
Returns time when last RRD archive was updated (all RRD files are considered).double
getPercentile(String sourceName)
Deprecated.UseVariable
based method instead.double
getPercentile(String sourceName, double percentile)
Deprecated.UseVariable
based method instead.int
getPixelCount()
Returns the number of pixels (target graph width).RrdDbPool
getPool()
(package private) Source
getSource(String sourceName)
String[]
getSourceNames()
Returns array of datasource names defined in this DataProcessor.long
getStep()
Returns the time step used for data processing.long[]
getTimestamps()
Returns consolidated timestamps created with theprocessData()
method.long[]
getTimestampsPerPixel()
Calculates timestamps which correspond to individual pixels on the graph based on the graph width set with asetPixelCount(int)
method call.long[]
getTimestampsPerPixel(int pixelCount)
Calculates timestamps which correspond to individual pixels on the graph.TimeZone
getTimeZone()
double[][]
getValues()
Returns an array of all datasource values for all datasources.double[]
getValues(String sourceName)
Returns calculated values for a single datasource.double[]
getValuesPerPixel(String sourceName)
Method used to calculate datasource values which should be presented on the graph based on the graph width set with asetPixelCount(int)
method call.double[]
getValuesPerPixel(String sourceName, int pixelCount)
Method used to calculate datasource values which should be presented on the graph based on the desired graph width.Variable.Value
getVariable(String sourceName)
Extract the variable value from an already define Variable datasource (a VDEF)Variable.Value
getVariable(String sourceName, Variable var)
Returns single aggregated value for a single datasource.boolean
isPoolUsed()
Returns boolean value representingRrdDbPool
usage policy.void
processData()
Method that should be called once all datasources are defined.void
setFetchRequestResolution(long fetchRequestResolution)
Sets desired RRD archive step in seconds to be used internally while fetching data from RRD files.void
setPixelCount(int pixelCount)
Sets the number of pixels (target graph width).void
setPool(RrdDbPool pool)
Defines theRrdDbPool
to use.void
setPoolUsed(boolean poolUsed)
Sets theRrdDbPool
usage policy.void
setStep(long step)
Roughly corresponds to the --step option in RRDTool's graph/xport commands.void
setTimeZone(TimeZone tz)
-
-
-
Field Detail
-
DEFAULT_PIXEL_COUNT
public static final int DEFAULT_PIXEL_COUNT
Constant representing the default number of pixels on a Rrd4j graph (will be used if no other value is specified withsetStep()
method.- See Also:
- Constant Field Values
-
DEFAULT_PERCENTILE
public static final double DEFAULT_PERCENTILE
ConstantDEFAULT_PERCENTILE=95.0
- See Also:
- Constant Field Values
-
DEFAULT_POOL_USAGE_POLICY
public static final boolean DEFAULT_POOL_USAGE_POLICY
Constant that defines the defaultRrdDbPool
usage policy. Defaults tofalse
(i.e. the pool will not be used to fetch data from RRD files)- See Also:
- Constant Field Values
-
-
Constructor Detail
-
DataProcessor
public DataProcessor(long t1, long t2)
Creates new DataProcessor object for the given time span. Ending timestamp may be set to zero. In that case, the class will try to find the optimal ending timestamp based on the last update time of RRD files processed with theprocessData()
method.- Parameters:
t1
- Starting timestamp in seconds without millisecondst2
- Ending timestamp in seconds without milliseconds
-
DataProcessor
public DataProcessor(Date d1, Date d2)
Creates new DataProcessor object for the given time span. Ending date may be set to null. In that case, the class will try to find optimal ending date based on the last update time of RRD files processed with theprocessData()
method.- Parameters:
d1
- Starting dated2
- Ending date
-
DataProcessor
public DataProcessor(Calendar gc1, Calendar gc2)
Creates new DataProcessor object for the given time span. Ending date may be set to null. In that case, the class will try to find optimal ending date based on the last update time of RRD files processed with theprocessData()
method. It use the time zone for starting calendar date.- Parameters:
gc1
- Starting Calendar dategc2
- Ending Calendar date
-
-
Method Detail
-
isPoolUsed
public boolean isPoolUsed()
Returns boolean value representingRrdDbPool
usage policy.- Returns:
- true, if the pool will be used internally to fetch data from RRD files, false otherwise.
-
setPoolUsed
public void setPoolUsed(boolean poolUsed)
Sets theRrdDbPool
usage policy.- Parameters:
poolUsed
- true, if the pool should be used to fetch data from RRD files, false otherwise.
-
getPool
public RrdDbPool getPool()
-
setPool
public void setPool(RrdDbPool pool)
Defines theRrdDbPool
to use. If not defined, but {setPoolUsed(boolean)
set to true, the defaultRrdDbPool.getInstance()
will be used.- Parameters:
pool
-
-
setPixelCount
public void setPixelCount(int pixelCount)
Sets the number of pixels (target graph width). This number is used only to calculate pixel coordinates for Rrd4j graphs (methods
The default number of pixels is defined by constantgetValuesPerPixel(String)
andgetTimestampsPerPixel()
), but has influence neither on datasource values calculated with theprocessData()
method nor on aggregated values returned fromgetAggregates(String)
and similar methods. In other words, aggregated values will not change once you decide to change the dimension of your graph.DEFAULT_PIXEL_COUNT
and can be changed with asetPixelCount(int)
method.- Parameters:
pixelCount
- The number of pixels. If you process RRD data in order to display it on the graph, this should be the width of your graph.
-
getPixelCount
public int getPixelCount()
Returns the number of pixels (target graph width). SeesetPixelCount(int)
for more information.- Returns:
- Target graph width
-
setStep
public void setStep(long step)
Roughly corresponds to the --step option in RRDTool's graph/xport commands. Here is an explanation borrowed from RRDTool:
"By default rrdgraph calculates the width of one pixel in the time domain and tries to get data at that resolution from the RRD. With this switch you can override this behavior. If you want rrdgraph to get data at 1 hour resolution from the RRD, then you can set the step to 3600 seconds. Note, that a step smaller than 1 pixel will be silently ignored."
I think this option is not that useful, but it's here just for compatibility.
- Parameters:
step
- Time step at which data should be fetched from RRD files. If this method is not used, the step will be equal to the smallest RRD step of all processed RRD files. If no RRD file is processed, the step will be roughly equal to the with of one graph pixel (in seconds).
-
getStep
public long getStep()
Returns the time step used for data processing. Initially, this method returns zero. OnceprocessData()
is finished, the method will return the real value used for all internal computations. Roughly corresponds to the --step option in RRDTool's graph/xport commands.- Returns:
- Step used for data processing.
-
getFetchRequestResolution
public long getFetchRequestResolution()
Returns desired RRD archive step (resolution) in seconds to be used while fetching data from RRD files. In other words, this value will used as the last parameter ofRrdDb.createFetchRequest()
method when this method is called internally by this DataProcessor.- Returns:
- Desired archive step (fetch resolution) in seconds.
-
setFetchRequestResolution
public void setFetchRequestResolution(long fetchRequestResolution)
Sets desired RRD archive step in seconds to be used internally while fetching data from RRD files. In other words, this value will used as the last parameter ofRrdDb.createFetchRequest()
method when this method is called internally by this DataProcessor. If this method is never called, fetch request resolution defaults to 1 (smallest possible archive step will be chosen automatically).- Parameters:
fetchRequestResolution
- Desired archive step (fetch resolution) in seconds.
-
getTimeZone
public TimeZone getTimeZone()
-
setTimeZone
public void setTimeZone(TimeZone tz)
-
getEndingTimestamp
public long getEndingTimestamp()
Returns ending timestamp. Basically, this value is equal to the ending timestamp specified in the constructor. However, if the ending timestamps was zero, it will be replaced with the real timestamp when theprocessData()
method returns. The real value will be calculated from the last update times of processed RRD files.- Returns:
- Ending timestamp in seconds
-
getTimestamps
public long[] getTimestamps()
Returns consolidated timestamps created with theprocessData()
method.- Returns:
- array of timestamps in seconds
-
getValues
public double[] getValues(String sourceName)
Returns calculated values for a single datasource. Corresponding timestamps can be obtained from thegetTimestamps()
method.- Parameters:
sourceName
- Datasource name- Returns:
- an array of datasource values
- Throws:
IllegalArgumentException
- Thrown if invalid datasource name is specified, or if datasource values are not yet calculated (methodprocessData()
was not called)
-
getAggregate
@Deprecated public double getAggregate(String sourceName, ConsolFun consolFun)
Deprecated.UseVariable
based method instead.Returns single aggregated value for a single datasource.- Parameters:
sourceName
- Datasource nameconsolFun
- Consolidation function to be applied to fetched datasource values. Valid consolidation functions are MIN, MAX, LAST, FIRST, AVERAGE and TOTAL (these string constants are conveniently defined in theConsolFun
class)- Returns:
- a aggregate value as a double calculated from the source.
- Throws:
IllegalArgumentException
- Thrown if invalid datasource name is specified, or if datasource values are not yet calculated (methodprocessData()
was not called)
-
getAggregates
@Deprecated public Aggregates getAggregates(String sourceName)
Deprecated.UseVariable
based method instead.Returns all (MIN, MAX, LAST, FIRST, AVERAGE and TOTAL) aggregated values for a single datasource.- Parameters:
sourceName
- Datasource name- Returns:
- Object containing all aggregated values
- Throws:
IllegalArgumentException
- Thrown if invalid datasource name is specified, or if datasource values are not yet calculated (methodprocessData()
was not called)
-
getVariable
public Variable.Value getVariable(String sourceName)
Extract the variable value from an already define Variable datasource (a VDEF)- Parameters:
sourceName
- Datasource name- Returns:
- A combined time and value extracted calculated from the datasource
-
getVariable
public Variable.Value getVariable(String sourceName, Variable var)
Returns single aggregated value for a single datasource.- Parameters:
sourceName
- Datasource namevar
- variable that will generate value- Returns:
- A combined time and value extracted calculated from the datasource
-
get95Percentile
@Deprecated public double get95Percentile(String sourceName)
Deprecated.UseVariable
based method instead.This method is just an alias forgetPercentile(String)
method. Used by ISPs which charge for bandwidth utilization on a "95th percentile" basis.The 95th percentile is the highest source value left when the top 5% of a numerically sorted set of source data is discarded. It is used as a measure of the peak value used when one discounts a fair amount for transitory spikes. This makes it markedly different from the average.
- Parameters:
sourceName
- Datasource name- Returns:
- 95th percentile of fetched source values
-
getPercentile
@Deprecated public double getPercentile(String sourceName)
Deprecated.UseVariable
based method instead.Used by ISPs which charge for bandwidth utilization on a "95th percentile" basis.The 95th percentile is the highest source value left when the top 5% of a numerically sorted set of source data is discarded. It is used as a measure of the peak value used when one discounts a fair amount for transitory spikes. This makes it markedly different from the average.
- Parameters:
sourceName
- Datasource name- Returns:
- 95th percentile of fetched source values
-
getPercentile
@Deprecated public double getPercentile(String sourceName, double percentile)
Deprecated.UseVariable
based method instead.The same asgetPercentile(String)
but with a possibility to define custom percentile boundary (different from 95).- Parameters:
sourceName
- Datasource name.percentile
- Boundary percentile. Value of 95 (%) is suitable in most cases, but you are free to provide your own percentile boundary between zero and 100.- Returns:
- Requested percentile of fetched source values
-
getSourceNames
public String[] getSourceNames()
Returns array of datasource names defined in this DataProcessor.- Returns:
- array of datasource names
-
getValues
public double[][] getValues()
Returns an array of all datasource values for all datasources. Each row in this two-dimensional array represents an array of calculated values for a single datasource. The order of rows is the same as the order in which datasources were added to this DataProcessor object.- Returns:
- All datasource values for all datasources. The first index is the index of the datasource,
the second index is the index of the datasource value. The number of datasource values is equal
to the number of timestamps returned with
getTimestamps()
method. - Throws:
IllegalArgumentException
- Thrown if invalid datasource name is specified, or if datasource values are not yet calculated (methodprocessData()
was not called)
-
addDatasource
public void addDatasource(String name, Plottable plottable)
Adds a custom,plottable
datasource (PDEF). The datapoints should be made available by a class extendingPlottable
class.- Parameters:
name
- source name.plottable
- class that extends Plottable class and is suited for graphing.
-
addDatasource
public void addDatasource(String name, String rpnExpression)
Adds complex source (CDEF). Complex sources are evaluated using the supplied
Complex sourceRPN
expression.name
can be used:- To specify sources for line, area and stack plots.
- To define other complex sources.
Rrd4j does not force you to specify at least one simple source name as RRDTool.
For more details on RPN see RRDTool's rrdgraph man page.
- Parameters:
name
- source name.rpnExpression
- RPN expression containing comma delimited simple and complex source names, RPN constants, functions and operators.
-
addDatasource
@Deprecated public void addDatasource(String name, String defName, ConsolFun consolFun)
Deprecated.UseVariable
based method instead.Adds static source (SDEF). Static sources are the result of a consolidation function applied to any other source that has been defined previously.- Parameters:
name
- source name.defName
- Name of the datasource to calculate the value from.consolFun
- Consolidation function to use for value calculation
-
addDatasource
@Deprecated public void addDatasource(String name, String sourceName, double percentile)
Deprecated.UseVariable
based method instead.Creates a datasource that performs a percentile calculation on an another named datasource to yield a single value. Requires that the other datasource has already been defined; otherwise, it'll end up with no data- Parameters:
name
- - the new virtual datasource namesourceName
- - the datasource from which to extract the percentile. Must be a previously defined virtual datasourcepercentile
- - the percentile to extract from the source datasource
-
addDatasource
public void addDatasource(String name, String defName, Variable var)
Creates a datasource that performs a variable calculation on an another named datasource to yield a single combined timestampe/value. Requires that the other datasource has already been defined; otherwise, it'll end up with no data- Parameters:
name
- - the new virtual datasource namedefName
- - the datasource from which to extract the percentile. Must be a previously defined virtual datasourcevar
- - a new instance of a Variable used to do the calculation
-
addDatasource
public void addDatasource(String name, String file, String dsName, ConsolFun consolFunc)
Adds simple datasource (DEF). Simple source
name
can be used:- To specify sources for line, area and stack plots.
- To define complex sources
- Parameters:
name
- source name.file
- Path to RRD file.dsName
- Datasource name defined in the RRD file.consolFunc
- Consolidation function that will be used to extract data from the RRD
-
addDatasource
@Deprecated public void addDatasource(String name, String file, String dsName, ConsolFun consolFunc, String backend)
Deprecated.Adds simple source (DEF). Source
name
can be used:- To specify sources for line, area and stack plots.
- To define complex sources
- Parameters:
name
- Source name.file
- Path to RRD file.dsName
- Data source name defined in the RRD file.consolFunc
- Consolidation function that will be used to extract data from the RRD file ("AVERAGE", "MIN", "MAX" or "LAST" - these string constants are conveniently defined in theConsolFun
class).backend
- Name of the RrdBackendFactory that should be used for this RrdDb.
-
addDatasource
public void addDatasource(String name, String file, String dsName, ConsolFun consolFunc, RrdBackendFactory backend)
Adds simple source (DEF). Source
name
can be used:- To specify sources for line, area and stack plots.
- To define complex sources
- Parameters:
name
- Source name.file
- Path to RRD file.dsName
- Data source name defined in the RRD file.consolFunc
- Consolidation function that will be used to extract data from the RRD file ("AVERAGE", "MIN", "MAX" or "LAST" - these string constants are conveniently defined in theConsolFun
class).backend
- Name of the RrdBackendFactory that should be used for this RrdDb.
-
addDatasource
public void addDatasource(String name, FetchData fetchData)
Adds DEF datasource with datasource values already available in the FetchData object. This method is used internally by Rrd4j and probably has no purpose outside of it.- Parameters:
name
- Source name.fetchData
- Fetched data containing values for the given source name.
-
addDatasource
public void addDatasource(String name, String dsName, FetchData fetchData)
Adds DEF datasource with datasource values already available in the FetchData object. This method is used internally by Rrd4j and probably has no purpose outside of it. The values will be extracted from dsName in fetchData.- Parameters:
name
- Source name.dsName
- Source name in the fetch data.fetchData
- Fetched data containing values for the given source name.
-
processData
public void processData() throws IOException
Method that should be called once all datasources are defined. Data will be fetched from RRD files, RPN expressions will be calculated, etc.- Throws:
IOException
- Thrown in case of I/O error (while fetching data from RRD files)
-
getValuesPerPixel
public double[] getValuesPerPixel(String sourceName, int pixelCount)
Method used to calculate datasource values which should be presented on the graph based on the desired graph width. Each value returned represents a single pixel on the graph. Corresponding timestamp can be found in the array returned fromgetTimestampsPerPixel()
method.- Parameters:
sourceName
- Datasource namepixelCount
- Graph width- Returns:
- Per-pixel datasource values
- Throws:
IllegalArgumentException
- Thrown if datasource values are not yet calculated (methodprocessData()
was not called)
-
getValuesPerPixel
public double[] getValuesPerPixel(String sourceName)
Method used to calculate datasource values which should be presented on the graph based on the graph width set with asetPixelCount(int)
method call. Each value returned represents a single pixel on the graph. Corresponding timestamp can be found in the array returned fromgetTimestampsPerPixel()
method.- Parameters:
sourceName
- Datasource name- Returns:
- Per-pixel datasource values
- Throws:
IllegalArgumentException
- Thrown if datasource values are not yet calculated (methodprocessData()
was not called)
-
getTimestampsPerPixel
public long[] getTimestampsPerPixel(int pixelCount)
Calculates timestamps which correspond to individual pixels on the graph.- Parameters:
pixelCount
- Graph width- Returns:
- Array of timestamps
-
getTimestampsPerPixel
public long[] getTimestampsPerPixel()
Calculates timestamps which correspond to individual pixels on the graph based on the graph width set with asetPixelCount(int)
method call.- Returns:
- Array of timestamps
-
dump
public String dump()
Dumps timestamps and values of all datasources in a tabular form. Very useful for debugging.- Returns:
- Dumped object content.
-
getLastRrdArchiveUpdateTime
public long getLastRrdArchiveUpdateTime()
Returns time when last RRD archive was updated (all RRD files are considered).- Returns:
- Last archive update time for all RRD files in this DataProcessor
-
-