What is a good solution for calculating an average where the sum of all values exceeds a double’s limits?

You can calculate the mean iteratively. This algorithm is simple, fast, you have to process each value just once, and the variables never get larger than the largest value in the set, so you won’t get an overflow. double mean(double[] ary) { double avg = 0; int t = 1; for (double x : ary) … Read more

3D Least Squares Plane

If you have n data points (x[i], y[i], z[i]), compute the 3×3 symmetric matrix A whose entries are: sum_i x[i]*x[i], sum_i x[i]*y[i], sum_i x[i] sum_i x[i]*y[i], sum_i y[i]*y[i], sum_i y[i] sum_i x[i], sum_i y[i], n Also compute the 3 element vector b: {sum_i x[i]*z[i], sum_i y[i]*z[i], sum_i z[i]} Then solve Ax = b for the … Read more

Fitting polynomials to data

Thanks for everyone’s replies. Here is another attempt at summarizing them. Pardon if I say too many “obvious” things: I knew nothing about least squares before, so everything was new to me. NOT polynomial interpolation Polynomial interpolation is fitting a polynomial of degree n given n+1 data points, e.g. finding a cubic that passes exactly … Read more

confidence and prediction intervals with StatsModels

For test data you can try to use the following. predictions = result.get_prediction(out_of_sample_df) predictions.summary_frame(alpha=0.05) I found the summary_frame() method buried here and you can find the get_prediction() method here. You can change the significance level of the confidence interval and prediction interval by modifying the “alpha” parameter. I am posting this here because this was … Read more