About: Numeric precision in Microsoft Excel

Facets (new session)
Description
Metadata
Settings
- Rule:
- Inverse Functional Properties:
- "Same As":

About: Numeric precision in Microsoft Excel Goto Sponge NotDistinct Permalink

An Entity of Type : owl:Thing, within Data Space : el.dbpedia.org associated with source document(s)

As with other spreadsheets, Microsoft Excel works only to limited accuracy because it retains only a certain number of figures to describe numbers (it has limited precision). With some exceptions regarding erroneous values, infinities, and denormalized numbers, Excel calculates in double-precision floating-point format from the IEEE 754 specification (besides numbers, Excel uses a few other data types). Although Excel can display 30 decimal places, its precision for a specified number is confined to 15 significant figures, and calculations may have an accuracy that is even less due to five issues: round off, truncation, and binary storage, accumulation of the deviations of the operands in calculations, and worst: cancellation at subtractions resp. 'Catastrophic cancellation' at subtraction

Attributes	Values
rdfs:label	Numeric precision in Microsoft Excel (en)
rdfs:comment	As with other spreadsheets, Microsoft Excel works only to limited accuracy because it retains only a certain number of figures to describe numbers (it has limited precision). With some exceptions regarding erroneous values, infinities, and denormalized numbers, Excel calculates in double-precision floating-point format from the IEEE 754 specification (besides numbers, Excel uses a few other data types). Although Excel can display 30 decimal places, its precision for a specified number is confined to 15 significant figures, and calculations may have an accuracy that is even less due to five issues: round off, truncation, and binary storage, accumulation of the deviations of the operands in calculations, and worst: cancellation at subtractions resp. 'Catastrophic cancellation' at subtraction (en)
sameAs	Numeric precision in Microsoft Excel
dbp:wikiPageUsesTemplate	dbt:Expand_section dbt:Reflist
Subject	Spreadsheet software Microsoft software Numerical analysis
prov:wasDerivedFrom	http://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft_Excel?oldid=1069137306&ns=0
Wikipage page ID	27918833 (xsd:integer)
page length (characters) of wiki page	24043 (xsd:nonNegativeInteger)
Wikipage revision ID	1069137306 (xsd:integer)
Link from a Wikipage to another Wikipage	Numerical integration Numerical methods for ordinary differential equations Round-off error Binary number Taylor series Byte Spreadsheet software Microsoft software Numerical analysis Double-precision floating-point format Micah Altman Quadratic equation Quadratic formula Runge–Kutta methods Truncation Truncation error Microsoft Excel Visual Basic for Applications IEEE 754-2008 revision Schrödinger equation Negative-feedback amplifier Significant figures Simpson's rule
has abstract	As with other spreadsheets, Microsoft Excel works only to limited accuracy because it retains only a certain number of figures to describe numbers (it has limited precision). With some exceptions regarding erroneous values, infinities, and denormalized numbers, Excel calculates in double-precision floating-point format from the IEEE 754 specification (besides numbers, Excel uses a few other data types). Although Excel can display 30 decimal places, its precision for a specified number is confined to 15 significant figures, and calculations may have an accuracy that is even less due to five issues: round off, truncation, and binary storage, accumulation of the deviations of the operands in calculations, and worst: cancellation at subtractions resp. 'Catastrophic cancellation' at subtraction of values with similar magnitude. (en)
foaf:isPrimaryTopicOf	http://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft_Excel
is Wikipage redirect of	Numerical precision in Microsoft Excel Accuracy and rounding in Microsoft Excel
is Link from a Wikipage to another Wikipage of	List of numerical analysis topics Numerical precision in Microsoft Excel Accuracy and rounding in Microsoft Excel

Faceted Search & Find service v1.17_git151 as of Feb 20 2025

Alternative Linked Data Documents: ODE Content Formats:

RDF

ODATA

Microdata

About

OpenLink Virtuoso version 07.20.3240 as of Nov 11 2024, on Linux (x86_64-ubuntu_focal-linux-gnu), Single-Server Edition (71 GB total memory, 1 GB memory in use)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2025 OpenLink Software