Change in default behavior of np.polyfit

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Change in default behavior of np.polyfit

Andreas Nußbaumer
Hi,

in [1] the scaling factor for the covariance matrix of `np.polyfit` was discussed. The conclusion was, that it is non-standard and a patch might be in order to correct this. Pull request [2] changes the factor from chisq(popt)/(M-N-2) to chisq(popt)/(M-N) (with M=number of point, N=number of parameters) essentially removing the "-2". Clearly, this changes the result for the covariance matrix (but not the result for the polynomial coefficients) and therefore the current behavior if `cov=True` is set.

It should be noted, that `scipy.optimize.curve_fit` also uses the chisq(popt)/(M-N) as scaling factor (without "-2"). Therefore, the change would remove a discrepancy.

Additionally, patch [2] adds an option that sets the scaling factor of the covariance matrix to 1 . This can be useful in occasions, where the weights are given by 1/sigma with sigma being the (known) standard errors of (Gaussian distributed) data points, in which case the un-scaled matrix is already a correct estimate for the covariance matrix. 

Best,
Andreas

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Change in default behavior of np.polyfit

Marten van Kerkwijk
Hi Andreas,

Thanks for noticing and correcting this unexpected scaling! The addition to get the unscaled version is also very welcome.

All the best,

Marten

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion