Statistics Tip of the Week: In Simple Nonlinear Regression, use a polynomial if the curve changes direction.

The "Simple" in "Simple Nonlinear" means that there is only one x Variable in the formula  of the formula e.g. y = f(x). The "nonlinear" means that we have determined that a straight line will not fit the data. We need to use some kind of curve -- e.g. Exponential, Logarithmic, Power, Polynomial, or some other type.

A Polynomial has a formula

Note that there is just one x Variable, but it is raised to various powers, starting with the power of 2. (If there were only a power of 1, the equation would be that of a straight line.)  The b's are Coefficients and the a is an Intercept.
​
A "2nd degree", also known as "2nd order" or "Quadratic", Polynomial is of the form:

A 2nd order Polynomial has 1 change in direction. As x increases, y increases and then decreases (or y decreases and then increases). Two examples are pictured above. These shapes are Parabolas.

A "3rd degree", aka "3rd order" aka Cubic" Polynomial has an x cubed term and changes direction twice.

A kth degree Polynomial has k – 1 changes in direction.

Simpler is better. It is usually not necessary to go beyond 3 orders. Larger orders are harder to work with. Also, they may be too closely associated with the idiosyncracies of the data provided in a particular Sample, and they may not be generally applicable to  data in other Samples from the same Population or Process. 

Reproduced by permission of John Wiley and Sons, Inc
from the book, Statistics from A to Z -- Confusing Concepts Clarified