How to set performance goals in Validation Manager: absolute, relative and range-specific goals

In Validation Manager, it is only possible to have one performance goal per parameter for a specific concentration level. It’s not always evident how to best use the goals, so when discussing the needs, it would be important to thoroughly find out, what actually is the need that needs to be met. Below some examples.

Smart use of performance goals helps laboratories assess verification results with clarity and confidence.

Key takeaways

• A single bias goal may not be suitable across the entire measuring range.

• Absolute goals may be more appropriate at lower concentrations, while relative goals may suit higher concentrations.

• Calculate the switch point as: 100 × absolute goal ÷ relative goal (%)

Explore how Validation Manager helps laboratories streamline verifications and validations.

Rule: “If the assay is within the % OR within in the absolute value, then it passes.”

Due to realities related measurements, it is common that there is no sensible way of setting one constant goal for the whole measuring range.

First of all, when we talk about random error in measurement, in many detection technologies there are both a constant component and a relative component in the random error that affects the measured results, and at low levels the constant component dominates the error related to the measurement, while on high levels the relative component dominates. This is visualized in the image below, where the dots represent values measured from samples of different concentrations, and the dashed lines delimit the area within which the measured results fall.

Second, when we do a comparison between two measurement procedures (or between measured values and the true values) and use a linear regression model to fit a line to the data set to represent the relation between the two measurement procedures, the equation we get is of form

y = intercept + slope * x.

Bias is the distance between the regression line and the y=x line, giving us basically

bias = intercept + ( slope – 1 ) * x.

If the measurement procedure under verification gives results from a wide range of concentrations, at low concentrations the intercept dominates the calculated bias, while on high concentrations the slope dominates the bias. For example, if our intercept is 0.005 g/L and our slope is 1.01, and we have measured results from 0.01 g/L to 100 g/L, at 0.01 g/L we have a bias of 0.0051 g/L or 51%, and at 100 g/L we have a bias of 1.005 g/L or 1.005%. This is visualized in the image below.

What this means for setting goals is that you might not be able to use one goal for the whole measuring range. With the values in our example above, expecting less than 0.006 g/L bias for 100 g/L samples could be both unnecessary and unrealistic, while allowing 1.005 g/L bias for levels around 0.01 g/L might not be clinically acceptable. And similarly, allowing 51% bias for values around 100 g/L probably wouldn’t be clinically acceptable, while it would be both unnecessary and unrealistic to demand anything better from levels around 0.01 g/L.

For these reasons, analytes with a wide measuring range easily require two goals. For example, if we state that the results are ok if on every concentration level the bias is either below 0.006 g/L or 2%, that would make our example above pass. At 0.3 g/L, 2% bias would be 0.006 g/L. At values lower than 0.3 g/L, 0.006 g/L > 2%, and at values above 0.3 g/L, 2% > 0.006 g/L. So, if we set our goal in Validation manager as 2%, and add a range-specific goal to compare results below 0.3 g/L to 0.006 g/L, that way we can ensure that all concentration levels will be below either of these two goals.

If you only have the goals, but you don’t know the level where to switch from one goal to another, it can be calculated as

100 * absolute goal (with unit) / relative goal (as %, without unit).

In our example above, 100 * 0.006 g/L / 2 = 0.3 g/L.

The limit can be calculated by adding your analytes and goals to columns B, D, E, and G in VM goal calculator.xlxs and then checking the limit from column O.

In Comparison studies using quantitative (numerical) results, the general goal related to an analyte is always used to assess the whole measuring range. When there are range-specific goals, with the default settings, if the range-specific goals pass, the conclusion is passed even if the general goal fails. (On the Goals page of your study Plan, you can select whether to require also the general goal to pass.) This means two things. First, if you want the automatic conclusion to consider both the absolute and the relative goal, you need to specify both of them as range-specific goals. Second, if you have a general goal, the color in the Overview table will show if it wasn’t met throughout the whole measuring range even if the conclusion is passed. This means that if you have your relative goal both as the general goal and as a range goal, and you also have the absolute goal as a range goal, this way the report will show you whether the relative goal passed or failed, even though the pass/fail conclusion is made based on the ranges.

In other studies that use range-specific goals, there can only be one goal per level, i.e. the general goal is considered on levels that are outside the defined ranges.

Rule: “If the assay is within the % goal throughout the whole measuring range, OR within the absolute goal throughout the whole measuring range, then it passes.”

Based on the reasoning above, please check if this really is what needs to be done. If it is, then it is recommended to do the study using the goal that’s more probable to be achieved. If the user doesn’t know which one to choose, then it’s recommended to use the relative goal. After the results have been imported into Validation Manager, go to the report and use the filters to show only failing reports. If needed, make notes of the results, for example by using the Comment feature in the Validation Manager detailed report. Open Validation Manager also in another browser window, and there, go to the study Plan Goals page, and change the goals for the failing analytes. Save the goals and wait for results being reanalyzed. Now, all the reports that fail, fail with both the relative goal and the absolute goal.

Rule: “An assay should preferably pass a goal set based on a biological variation database, but if it doesn’t, it’s enough to pass what manufacturer promises.”

If it’s important to check which analytes pass the stricter goal, plan the study using the stricter goals. After the results have been imported into Validation Manager, go to the report and use the filters to show only failing reports. If needed, make notes of the results, for example by using the Comment feature in the Validation Manager detailed report. Open Validation Manager also in another browser window, and there, go to the study Plan Goals page, and change the goals for the failing analytes. Save the goals and wait for results being reanalyzed. Now, all the reports that fail, fail both goals.

In a Comparison study using quantitative (numerical) results, it is optionally possible to use the range-specific goals to enable considering two goals at the same time. The general goal set for an analyte always considers the whole measuring range. In addition to that, you can have range-specific goals. Validation Manager assesses each range separately, whether they met their goals or not. With the default settings of the study, the decision of whether the analyte passes or not is only based on range-specific goals (and medical decision points) if such exist, but the color in the Overview table also shows whether the general goal was reached throughout the whole measuring range.

Streamline laboratory verifications and validations

Discover how Validation Manager helps laboratories automate workflows, reduce manual work and maintain complete traceability.

See how it works