Fits a nonlinear regression model or optimizes a scalar function.

### Options

`PRINT` = string tokens |
What to print (`model` , `deviance` , `summary` , `estimates` , `correlations` , `fittedvalues` , `accumulated` , `monitoring` , `grid` ); default `mode` , `summ` , `esti` or `grid` if `NGRIDLINES` is set |
---|---|

`CALCULATION` = expression structures |
Calculation of fitted values or of explanatory variates involving nonlinear parameters; default `*` (valid only if `OWN` set) |

`OWN` = scalar |
Option setting for `OWN` directive if this is to be used rather than `CALCULATE` ; default `*` requests `CALCULATE` to be used |

`CONSTANT` = string token |
How to treat the constant (`estimate` , `omit` ); default `esti` |

`FACTORIAL` = scalar |
Limit for expansion of model terms; default as in previous `TERMS` statement, or 3 if no `TERMS` given |

`POOL` = string token |
Whether to pool ss in accumulated summary between all terms fitted in a linear model (`yes` , `no` ); default `no` |

`DENOMINATOR` = string token |
Whether to base ratios in accumulated summary on rms from model with smallest residual ss or smallest residual ms (`ss` , `ms` ); default `ss` |

`NOMESSAGE` = string tokens |
Which warning messages to suppress (`dispersion` , `leverage` , `residual` , `aliasing` , `marginality` , `vertical` , `df` ); default `*` |

`FPROBABILITY` = string token |
Printing of probabilities for variance and deviance ratios (`yes` , `no` ); default `no` |

`SELECTION` = string tokens |
Statistics to be displayed in the summary of analysis produced by `PRINT=summary` , `seobservations` is relevant only for a Normally distributed response, and `%cv` only for a gamma-distributed response (`%variance` , `%ss` , `adjustedr2` , `r2` , `seobservations` , `dispersion` , `%cv` , `%meandeviance` , `%deviance` , `aic` , `bic` , `sic` ); default `%var` , `seob` if `DIST=normal` , `%cv` if `DIST=gamma` , and `disp` for other distributions |

`NGRIDLINES` = scalar |
Number of values of each parameter for a grid of function evaluations; default `*` |

`SELINEAR` = string token |
Whether to calculate s.e.s for linear parameters (`yes` , `no` ); default `no` |

`INOWN` = identifiers |
Setting to be used for the `IN` parameter of `OWN` if used in place of `CALCULATE` ; default `*` |

`OUTOWN` = identifiers |
Setting to be used for the `OUT` parameter of `OWN` if used in place of `CALCULATE` ; default `*` |

### Parameter

formula |
List of explanatory variates and/or one factor to be used in linear regression, within nonlinear optimization |
---|

### Description

`FITNONLINEAR`

can fit nonlinear regression models, or optimize a general function. However, you should check first that the model is not included in the linear and generalized linear models that can be fitted by `FIT`

or the standard curves provided by `FITCURVE`

, for these are much more efficient. Because the methods used by `FITNONLINEAR`

are very general, they are neither as robust nor as automatic as, for example, the method that is used for fitting linear models.

It is better also to use `FIT`

for nonlinear models with separable linear parameters (generalized nonlinear models), because there are fewer constraints on the model specification. `FIT`

differs from `FITNONLINEAR`

as follows. `FIT`

takes account of the settings of the `LINK`

and `EXPONENT`

options of `MODEL`

(but `FITNONLINEAR`

does not). Furthermore, in `FIT`

the parameter may be set to a general model formula, including more than one factor, interactions and functions. However, `FIT`

cannot be used for optimisation of general functions, nor for models with no linear parameters.

Nonlinear methods make use of iterative optimization algorithms, designed to search for the minimum value of a function as the parameters vary; for nonlinear regression models, the function involved is the deviance, or minus twice the log-likelihood ratio, so the algorithm searches for the maximum-likelihood solution. It is often necessary to provide the algorithm with good starting values, to set bounds on the parameter values, and sometimes even to define the initial direction of search. Optimization is easiest with few parameters, approximately quadratic functions, small correlations between parameters, and good initial parameter estimates. Where possible, you can effectively reduce the number of parameters to be optimized by separating linear and nonlinear ones: that is, you can first fit the linear parameters, and treat the resulting residual sums of squares as functions of the nonlinear parameters alone. Problems with optimization methods are most likely to arise if you neglect the parameterization of the function. You can often transform the parameters to make the function nearly quadratic; after finding a solution, you can then use the `RFUNCTION`

directive to estimate the original parameters. Another source of difficulty is if you try to fit inappropriately many parameters. For advice on reformulating functions to speed up optimization, see Ross (1990).

Before using `FITNONLINEAR`

you must use the `MODEL`

directive with either the `Y`

parameter set to response variate, or the `FUNCTION`

option set to a scalar that is to store the value of a general function to be optimized. You must also use the `RCYCLE`

directive to specify the nonlinear parameters. The `TERMS`

directive can be used as in linear regression, to list the explanatory variables to be used in modelling. The model calculations themselves are provided in expression structures which are supplied by the `CALCULATION`

option of `FITNONLINEAR`

. You can modify the model using the `ADD`

, `DROP`

and `SWITCH`

directives as usual, except that you must give a `TERMS`

statement first. You can use the `RDISPLAY`

and `RKEEP`

directives to display or save the results. The `RCHECK`

procedure can be used to assess the fit of the model, and `RGRAPH`

can display the fit with respect to some specified variate.

Genstat fits nonlinear regression models by maximum likelihood. The likelihood is usually from a distribution in the exponential family; this is specified using the `DISTRIBUTION`

option of the `MODEL`

directive. The settings of the `LINK`

and `EXPONENT`

options of the `MODEL`

directive are ignored, and you are not allowed to set the `GROUPS`

option; other options and parameters are as in linear regression.

The `RCYCLE`

directive allows you to select the algorithm (of the three available) to be used to fit the model; these work with numerical differences and so do not require you to specify derivatives. The default algorithm is a modified Gauss-Newton method. This takes advantage of the fact that the likelihood function can be expressed as a sum of squares. However, you cannot use it for minimizing a general function. The second algorithm, a modified Newton method, is requested by setting option `METHOD=Newton`

in the `RCYCLE`

statement. This can be used for any nonlinear model. The third algorithm is a modified Fletcher-Powell method, specified by setting `METHOD=Fletcher`

. In fact, this is similar to the Newton method, with an occasional step in the search being determined by the Fletcher-Powell algorithm rather than by the Newton algorithm. The modification in all these methods is to use estimated numerical differences instead of evaluating derivatives. In nonlinear regression problems, particularly ones with separable linear parameters, specification of the derivatives would be very complex, and so it is much more convenient to estimate them numerically.

You must set the `PARAMETER`

parameter of `RCYCLE`

to the identifiers of scalars that will be used to represent the nonlinear parameters in the model calculations. There must be at least one nonlinear parameter. There is no formal upper limit on the number of nonlinear parameters, but the greater the number of parameters the longer the time required for the search and the smaller the chance of finding a satisfactory solution.

You can set the `LOWER`

and `UPPER`

parameters of `RCYCLE`

to provide fixed bounds for each parameter. By default, the values +10^{9} are used. Where possible you should always set bounds, particularly to avoid such problems as attempting to take the log of a negative number. You can incorporate more general constraints as logical functions within the calculations. For example you could compute an extra term

`(Constr > 0) * K * Constr`

to impose a penalty on exceeding the constraint, controlled by setting different values of `K`

. Often, the best way to impose a constraint is to reparameterize. For example, if a parameter α must be positive, you could replace α by exp(β), and allow β to take any value.

The `STEPLENGTH`

parameter of `RCYCLE`

can be used to provide initial step lengths for the search. By default the step length is 0.05 times the initial value of the corresponding parameter, or precisely 1.0 if the initial value is zero. If you set a step length to zero, Genstat treats the corresponding parameter as being fixed at its initial value. This allows complex problems in many dimensions to be tackled in stages, optimizing some parameters with others fixed, and then optimizing the others in turn.

By default, the initial value of a parameter is taken to be the current value of the scalar that represents it in the calculation, or 1.0 if the value is missing. Other values can be specified using the `INITIAL`

parameter of `RCYCLE`

.

If you can calculate a range within which you expect a parameter to lie, you should choose a step length of about 1% of the width of the range. If the steps are too small, numerical differencing may not work; if they are too large, gradients may be unreliable and you may get premature convergence. Genstat tests convergence by the relationship of final adjustments to step lengths.

The more parameters there are to estimate, and the more scattered are the data, the more iterations are required to find the optimum. The maximum number of iterations is set to 30 by default, but you can reset this with the `MAXCYCLE`

option of `RCYCLE`

. However, if convergence fails with a given setting of `MAXCYCLE`

, you should check the data and consider reparameterizing the model before you indiscriminately increase the number of iterations.

Genstat prints a warning when convergence fails. The only sections of output that are then available are the residual degrees of freedom, the residual deviance, the fitted values, and the parameter estimates (without standard errors) for the current cycle. The `EXIT`

parameter of the `RKEEP`

directive allows you to obtain a numerical code indicating why convergence failed.

Many of the options of `FITNONLINEAR`

are the same as those of `FIT`

: `PRINT`

, `CONSTANT`

, `FACTORIAL`

, `POOL`

, `DENOMINATOR`

, `NOMESSAGE`

, `FPROBABILITY`

and `SELECTION`

are all as in `FIT`

. The `grid`

setting of `PRINT`

is used with the `NGRIDLINES`

option. If you set `NGRIDLINES`

to *n*, say (with *n*≥2), `FITNONLINEAR`

evaluates the likelihood at a grid of values of the nonlinear parameters, and does not search for an optimum. For each parameter, the distance between the upper and lower bounds (set by the `RCYCLE`

directive) will be divided into (*n*-1) equal parts, defining a rectangular grid with *n* gridlines in each dimension. By setting some upper and lower bounds equal, you can look at the behaviour of the function with respect to a few parameters at a time. The default setting of the `PRINT`

option is `grid`

in this case, and produces a display of the function values. Other settings of the `PRINT`

option are then ignored. The calculated grid of values is available from the `GRID`

parameter of the `RKEEP`

directive, and can be used to produce pictures of the surface for example with the `DCONTOUR`

or `DSURFACE`

directives.

You must set one of the `CALCULATION`

and `OWN`

options to define how the nonlinear parameters are included in the model. The `CALCULATION`

option does this by a list of one or more expressions. The expressions are evaluated in turn at every step of the estimation process, just as if they had been given in a sequence of `CALCULATE`

statements. For example:

`EXPRESSION Diffuse[1];\`

` VALUE=!E(Xl,Xr=NORMAL((H+1,-1*X)/SQRT(2*D*T))`

`& Diffuse[2]; VALUES=!E(Z=Xl+Xr-1)`

`FITNONLINEAR [CALCULATION=Diffuse[1,2]] Z`

Here, the `CALCULATION`

option is set to the two expressions `Diffuse[1]`

and `Diffuse[2]`

, to define a model for one-dimensional diffusion. Alternatively, you can set the `OWN`

option to specify that the calculation is to be done by executing your own source code, called by a version of the subroutine G5XZXO, as for the `OWN`

directive. Generally, using `OWN`

is likely to be worthwhile only when calculations are very extensive, or when a particular function is needed often. The setting of the `OWN`

option will be passed to G5XZXO in the same way as the setting of the `SELECT`

option of the `OWN`

directive is passed to G5XZXO. The `INOWN`

and `OUTOWN`

options then define data structures to provide the input and store the output from `FITNONLINEAR`

.

There are three ways of using `FITNONLINEAR`

. The first provides the most efficient method when the model is linear in some of the parameters. However, this can be used only if the data are Normally distributed, or if they follow a Poisson distribution and the model contains only one explanatory variable and no constant term. The linear parameters are fitted by a linear regression of the response variate (specified by the parameter of the `MODEL`

statement) on the variates listed by the parameter of `FITNONLINEAR`

. At least one of these variates must depend on the nonlinear parameters in the model but they need not all do so. You can define how to calculate the variates from the nonlinear parameters either by the `CALCULATION`

option or by the `OWN`

, `INOWN`

and `OUTOWN`

options of `FITNONLINEAR`

. The parameter of `FITNONLINEAR`

may include variates that are not changed by the calculations as well as those that are. One factor may also be included so that a separate constant is fitted for each level, giving a set of parallel nonlinear regressions. You cannot include interactions between a variate and a factor, as is allowed with `FIT`

or `FITCURVE`

; nor can you include `POL`

, `REG`

, `COMPARISON`

, `SSPLINE`

or `LOESS`

functions, nor interactions between variates as allowed with `FIT`

. However, procedure `FITPARALLEL`

allows you to assess the various ways in which nonlinear models can be non-parallel. If there is a constant in the linear regression, as specified by the `CONSTANT`

option, the factor will be parameterized in terms of differences from the first level – as in linear regression. If you set `CONSTANT=omit`

, the actual constants are fitted; there is no need to set option `FULL`

of the `TERMS`

directive which is ignored in nonlinear models. If you have specified an offset variate using the `MODEL`

directive, its values can also be modified by the calculations, and depend on the parameters. By default, standard errors are calculated only for nonlinear parameters. To obtain standard errors for the linear parameters as well, you can set option `SELINEAR=yes`

. Then, after the optimum has been found, Genstat increases the number of dimensions to include the linear parameters and estimates the rate of change of the likelihood in all the dimensions.

If there are no linear parameters in the model, or if the distribution is not Normal or Poisson, you should no longer use the parameter of `FITNONLINEAR`

. Instead you should set the `FITTEDVALUES`

parameter in the `MODEL`

statement to the identifier of a variate that is to contain the fitted values for any set of values of the nonlinear parameters. Then define how to calculate the fitted values from the nonlinear parameters and the explanatory variates, using either the `CALCULATION`

or the `OWN`

option of `FITNONLINEAR`

. The distribution can now be any of those available from the `DISTRIBUTION`

option of the `MODEL`

directive, taking account of the settings of the `DISPERSION`

and `WEIGHTS`

options of the `MODEL`

directive. The multinomial distribution is used rather differently from the others: it is for fitting distributions. The `DISTRIBUTION`

directive provides a wide range of standard distributions, and is more convenient and efficient than `FITNONLINEAR`

for these; but `FITNONLINEAR`

allows you to fit other distributions. To specify and fit your own distribution, you should supply as response variate a set of counts of observations falling into a series of groups; the fitted values should then be a set of expected counts for the groups, calculated from the distribution being considered. The resulting multinomial likelihood is the same as that of the Poisson distribution, but with the constraint Σf_{i}=*M*, where *M* is the sum of the counts.

The third method allows you to minimize a general function. (You can still use this to fit statistical models by supplying the deviance, which is minus twice the log-likelihood ratio.) To minimize a function, you need to start with a `MODEL`

statement that has no response variate, but where the `FUNCTION`

option is set to a scalar. You then specify the parameters with the `RCYCLE`

directive as before, and perform the minimization with `FITNONLINEAR`

, supplying an expression that calculates the function from the parameters and places the result into the scalar. When you are minimizing a general function in this way, some of the output from `FITNONLINEAR`

is different. Genstat ignores the `accumulated`

and `fittedvalues`

settings of the `PRINT`

option, and the `deviance`

and `summary`

settings display only the minimum function value. The `correlation`

setting displays the inverse of the estimated matrix of second derivatives of the function with respect to the parameters, scaled by the diagonal values. Similarly, in place of the standard errors usually displayed by the `estimates`

setting, Genstat prints the square roots of the diagonal values of twice the inverse of the second-derivative matrix. These can give a useful indication of the form of the function near the minimum. If the function is a deviance you can interpret these as asymptotic standard errors and correlations (not scaled by an estimate of dispersion). For a general function, the “s.e.” can be interpreted as the approximate change in a parameter required to increase the function by 1.0 starting from the minimum. Genstat ignores the `CONSTANT`

option of the `FITNONLINEAR`

directive for general functions, and you must not set the parameter. Similarly, the `WEIGHTS`

and `OFFSET`

options of the `MODEL`

directive are ignored, and the `GROUPS`

option must not be set. The only parameters of the `RKEEP`

directive that are available are `ESTIMATES`

, `SE`

, `INVERSE`

, `EXIT`

, `GRADIENTS`

and `GRID`

. The minimum value of the function is of course available in the scalar specified by the `FUNCTION`

option of the `MODEL`

directive.

Options: `PRINT`

, `CALCULATION`

, `OWN`

, `CONSTANT`

, `FACTORIAL`

, `POOL`

, `DENOMINATOR`

, `NOMESSAGE`

, `FPROBABILITY`

, `SELECTION`

, `NGRIDLINES`

, `SELINEAR`

, `INOWN`

, `OUTOWN`

.

Parameter: unnamed.

### Action with `RESTRICT`

You can restrict the units that Genstat will use for fitting the model by putting a restriction on any of the vectors involved in the `MODEL`

statement (response variate, weight variate, offset variate or variate of binomial totals), or on any explanatory variate or factor. However, you are not allowed to have different restrictions on the different vectors.

### Reference

Ross, G.J.S. (1990). *Nonlinear Estimation*. Springer-Verlag, New York.

### See also

Directives: `EXPRESSION`

, `MODEL`

, `RDISPLAY`

, `RKEEP`

, `RCYCLE`

, `RFUNCTION`

, `FIT`

, `FITCURVE`

.

Procedures: `NLAR1`

, `HGNONLINEAR`

, `RQUADRATIC`

, `MINIMIZE`

, `MIN1DIMENSION`

, `SIMPLEX`

.

Commands for: Regression analysis.

### Example

" Example FITN-1: Fitting a standard curve using FITNONLINEAR Modelling the relationship between yield of sugar from cane and amount of nitrogen fertilizer. NOTE: this model is fitted most efficiently in Genstat by the FITCURVE directive with option CURVE=exponential. This example is to illustrate the use of FITNONLINEAR, which is really provided for fitting models that are not in FITCURVE." READ Nitrogen,Yield 0 60 0 73 0 77 0 72 50 125 50 144 50 145 50 116 100 152 100 154 100 160 100 141 150 182 150 167 150 181 150 185 200 198 200 188 200 189 200 182 : " 1) Three parameters treated as nonlinear" MODEL Yield; FITTED=Expon RCYCLE PARAMETER=Rate,Range,Inter; INITIAL=0.95,-100,100;\ UPPER=1,*,*; STEP=0.01,*,* EXPRESSION [VALUE=(Expon=Inter+Range*Rate**Nitrogen)] Calcexp FITNONLINEAR [CALCULATION=Calcexp] " Display the fitted curve. Note that the parameter of the RGRAPH procedure must be set to the x-variate to be used in the plot after a nonlinear fit." RGRAPH [GRAPHICS=high] Nitrogen " Plot residuals against fitted values. Note that the RCHECK procedure cannot be used to check assumptions after a nonlinear fit." RKEEP RESIDUAL=r; FITTED=f DGRAPH r; f " 2) Linear parameters separated" MODEL Yield RCYCLE PARAMETER=Rate; INITIAL=0.95; UPPER=1; STEP=0.01 EXPRESSION [VALUE=(Expon=Rate**Nitrogen)] Calcexp FITNONLINEAR [PRINT=model,summary,estimates,correl,fitted;\ CALCULATION=Calcexp] Expon " Estimate s.e.s for linear parameters as well" FITNONLINEAR [PRINT=m,s,e,c; CALCULATION=Calcexp; SELINEAR=yes] Expon " 3) Explicit likelihood" MODEL [FUNCTION=Sum] RCYCLE PARAMETER=Rate,Range,Inter; INITIAL=0.95,-100,100;\ UPPER=1,*,*; STEP=0.001,*,* EXPRESSION [VALUE=(Sum=SUM((Yield-Inter-Range*Rate**Nitrogen)**2))]\ Calcexp FITNONLINEAR [CALC=Calcexp] " To provide true standard errors, multiply SE by residual variance." RKEEP ESTIMATES=Estimates; SE=se CALC sigma2 = Sum / (NVALUES(Yield) - 3) & se = se * SQRT(sigma2) PRINT Estimates,se