One way to calculate an R² for a nonlinear model is demonstrated in the SAS/STAT Sample Library program NLINRSQ, titled "Computation of R-Square for a Non-Linear Model." However, you should note that this estimator is no longer bounded by zero and one. There is also debate as to whether corrected sums of squares should be used.

For a discussion of the properties of different estimators of R², see Kvalseth (1985) which discusses eight different estimators of R², all of which are equivalent for a linear model with an intercept in terms of both linear models without an intercept and nonlinear models.

Kvalseth, T.O. (1985), "Cautionary Note About R²," The American Statistician, 39, 279.