SOP for Determination of Significant figures and Rounding Rules
1.0 OBJECTIVE:
To provide a procedure for determination of significant figures and uniform rounding rules for the different mathematical calculations performed by operational departments like, Quality Control, Production, Engineering, Warehouse and Quality Assurance etc.
2.0 SCOPE:
The scope of the SOP provides the procedure for determination of significant figure and how to do the rounding for different calculations.
3.0 RESPONSIBILITY :
Personnel of concerned department to implement.
Heads of concerned departments to ensure compliance.
4.0 PROCEDURE :
This procedure is applicable to all mathematical calculations involved in Manufacturing, Packaging, testing, calibration etc. where tolerance limits are specified.
4.1 Definitions:
Rounding: Mathematical reduction of numerical values to the digit required according to the significant figure or tolerance established for the particular operation, method or process.
Significant figure: Number of digits in a decimal or whole number, to be considered as significant or representative of the real value, specification limit, or result for a particular method, operation or process.
Tolerance: Acceptable range established for a particular method, operation or process expressed as lower and upper limit. When the tolerances are numerically expressed the same is considered as significant to the last digit shown. This is valid regardless of whether the values are expressed as percentages or absolute numbers.
4.2 Significant figure determination:
4.2.1 The significant figure for a numerical value is normally given by the tolerance limit, established for the particular method, operation or process under consideration.
4.2.2 If the tolerance limit is expressed in decimal places, the result obtained must also be expressed or documented in decimal places following the same number of decimal places. If no decimal place is allowed, no decimal place is reported.
4.2.3 If it is a set value having no tolerances, (as in the case of a weighing process for a raw material) the process must be documented according to decimal places determined by the set value. The balance or any equipment used for the process must provide accuracy up to the last digit established by the current specifications.
4.2.4 In the case of testing methods, the significant figures are normally indicated by the methods itself. For Analytical methods follow the significant figures established by the current specifications.
4.3 Rounding:
4.3.1 The following rules are followed whenever rounding is needed to reduce a numerical expression or value to the last significant digit.
4.3.2 Determine the number of significant digits to be used in reported result.
4.3.3 When rounding off is required, consider only one digit in decimal place to the right of the last place in limit expression.
4.3.4 If this digit is smaller than 5, it is eliminated and the preceding digit is unchanged. e.g. If that digit is 0, 1, 2, 3, or 4 – then the previous digit remains as it is.
4.3.5 If the digit is greater than 5, it is eliminated and the preceding digit is increased by one. e.g. If that digit is 6, 7, 8, or 9 – then the previous digit is increased by one.
4.3.6 If the digit equals to 5, the 5 is eliminated and the preceding digit is increased by one.
4.3.7 Intermediate calculation may be rounded for reporting purpose, but the original value (not rounded) should be used for any additional required calculation.
4.3.8 Rounding off should not be done until the final calculations for the reportable value have been completed.
4.3.9 In case any doubt on how to round a significant figure shall be consulted immediate senior .
Example Table:
4.4 Exceptions:
1) In case of reporting of values for limits < 0.5% or and < 0.1%, rounding off shall be done as described below in the below mentioned specific cases.
a) When the specification is a whole number with out any decimal and the calculated value is less than 0.5%, result shall be reported to one decimal after rounding off.
Example Table:
b) When the specification is in 1(one) decimal and the calculated value is less than 0.1%, result shall be reported to two decimal after rounding off.
Example Table:
2) For calculating the Assay on anhydrous basis, Moisture content/LOD shall be taken without rounding the calculated value, irrespective of the no. of significant figures present in the specification.
3) For reporting of assay and limit test following example is illustrated for rounding numerical values for comparison with requirements
Example Table:
5.0 ANNEXURES
Nil