Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [regression-analysis]

This tag is for questions about regression analysis. In statistical modeling, regression analysis is a statistical process for estimating the relationships among variables. It includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables (or 'predictors').

(Wikipedia) Regression analysis helps one understand how the typical value of the dependent variable (or 'criterion variable') changes when any one of the independent variables is varied, while the other independent variables are held fixed.

Most commonly, regression analysis estimates the conditional expectation of the dependent variable given the independent variables – that is, the average value of the dependent variable when the independent variables are fixed.

392 questions
16
votes
2 answers

Sxx in linear regression

What is the meaning of the symbols $S_{xx}$ and $S_{xy}$ in simple linear regression? I know the formula but what is the meaning of those symbols?
Dkova
  • 359
7
votes
2 answers

Update a regression on the fly?

Say I have 100 people each with a height, weight, and age. I make a regression that predicts age based on height and weight. Now, I would like to update that model when I meet someone new. I don't want to just re-process 101 people though--I want to…
Sam Weisenthal
  • 287
6
votes
1 answer

Proving Linear Regression by Using Physical Springs Model

I found a nice proof of the linear regression formulas by using physical springs in Mark Levi's Mathematical Mechanic on page 43. Linear Regression (The Best Fit) via Springs Imagine a collection of data points $(x_i, y_i)$ in the plane. We are…
Agile_Eagle
  • 3,016
6
votes
0 answers

single variable is significant but overall test is not

I do a multiple regression with 3 independent variables $X_1$, $X_2$ and $X_3$. The correlation between $Y$ and $X_1$, $Y$ and $X_2$, and $Y$ and $X_3$, are each large and statistically significant. When I fit the multiple regression, none of the…
Mike Brown
  • 492
6
votes
3 answers

How to prove SSE and SSR are independent

Consider $Y=X\beta+\varepsilon$, where $X$ is n by p, $\beta$ is p by 1 and $\varepsilon$ is n by 1 with covariance matrix = var($\varepsilon$)=$\sigma^2 I$. Give expression for the regression and error sums of squares, find their expected values,…
81235
  • 1,306
  • 11
  • 23
5
votes
1 answer

Sqrt LASSO vs LASSO

In the paper Square Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming they talk about Sqrt-LASSO which is simply just trying to minimize $\|Ax-b\|_2 + \lambda\|x\|_1$ rather than the regular LASSO $\|Ax-b\|_2^2 +…
jeffj1355
  • 73
5
votes
4 answers

Does least squares (approximate solution) minimize the orthogonal distance of $b$ to $Ax$, or does it minimize the error projected along the $b$ axis?

I have always been confused about whether the approximate solution to $Ax=b$ is equivalent to minimizing the average distance of all of the $b$ vectors to $Ax$, or whether it is minimizing the distance projected along the $b$ axis? (where $A$ is…
makansij
  • 1,643
5
votes
1 answer

Why does Desmos give a different exponential regression than my TI 84 and my Calculus text?

I obtained this data from a calculus text (Stewart's Single Variable Calculus, Early Transcendentals, 2nd Ed.). The text and my TI 84 graphing calculator both give similar values for $a$ and $b$ ($0.00807079266$ and $1.013731$, respectively). Below…
MathAdam
  • 3,397
4
votes
2 answers

Line of best fit for $\{(n,n+\sin n) : n \in \mathbb{Z}\}$

It seems intuitive that the line of best fit for $\{(n,n+\sin n) : n\in \mathbb{Z}\}$ should be $y=x$. More concretely, it seems like a reasonable conjecture would be: If $y = m_k x + b_k$ is the line of best fit for the set of points $$\{…
Patch
  • 4,607
4
votes
0 answers

finding optimal entry and exit points in a time series of prices

given an initial 1000 dollars cash value, and a time series of bid price, ask price, assuming constraints of transaction costs, e.g. 1 dollar every time we buy or sell, what is the best approach to finding buy and sell points that maximize the…
John
  • 103
4
votes
2 answers

How do I apply regression techniques to data I think fits a differential equation?

I have a calculus background and am beginning to learn about regression and statistical inference. Let's say I have some $(x,y)$ data and want to check how well it fits an ordinary differential equation with no closed-form solution such…
Raymond Hancock
  • 93
4
votes
1 answer

LASSO relationship between Lagrange multiplier and constraint and why it doesn't matter

My understanding of LASSO regression is that the regression coefficients are selected to solve the minimisation problem: $$\min_\beta \|y - X \beta\|_2^2 \ \\s.t. \|\beta\|_1 \leq t$$ In practice this is done using a Lagrange multiplier, making the…
ConstantAmateur
  • 141
4
votes
1 answer

Derivation of standard error of regression estimate with degrees of freedom

I am taking a course of Econometrics: I need help to understand as to how do we arrive at the formula for standard error of regression $$\hat{\sigma}^2=\frac{\sum{e_i^2}}{n-k}.$$ I understand the bessel's correction required to remove the bias…
Math Student
  • 65
4
votes
1 answer

sequential anova r

I am a really confused. Assume we have a multiple regression model: $$ y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 +\cdots+ \beta_k x_k $$ Using R we can make a test: $$ H0: \beta_1 = \beta_2 = \cdots = \beta_k\\ H1: \beta_j \neq 0 \; \text{for at…
Mr.M
  • 493
4
votes
1 answer

How many data points are "enough" for linear regression?

I have data points $(x_t,y_t)$ generated from $y_t = a + b x_t + \epsilon$ where $\epsilon$ is gaussian error term with zero mean and unknown variance. I want to estimate coefficients $a$ and $b$ but their is some cost associated with generating…
user2684957
  • 397
1
2 3
…
26 27