Solved

# Decimal precision in results grid

• 3 replies
• 36 views

All, we have fields for a graphic’s width and height. In the results grid I have a formula to calculate the Square Inches; however, it’s rounding to 0 precision (no decimal places). How do I extend this Square inches calculation to 3 places?

``=CInt(CInventoryItem.UsrCFCHeight] * tInventoryItem.UsrCFCWidth])``

### 3 replies

Userlevel 7
+19

@swartzfeger   In the above formula, you're using CINT, which converts to an Integer type and won't include any decimal places. This conversion isn't necessary if you're using default values of 0.00 for those fields.

If you still need to perform a conversion for some reason, please use the CDEC/CDBL function instead and check the results.

Below is the screenshot for reference.

Userlevel 3

Try CDbl instead of CInt. Integers don’t like dots.

You could also try the Round function under Math. Round( [insert calculation] , 2 ) should give you at least a precision of two.

Userlevel 5

Try CDbl instead of CInt. Integers don’t like dots.

You could also try the Round function under Math. Round( [insert calculation] , 2 ) should give you at least a precision of two.

Thank you Naveen and Martin, CDbl worked! (I should’ve known this, Data Types 101!)