This tag is for questions about stochastic approximation which are a family of methods of iterative stochastic optimization algorithms that attempt to find zeroes or extrema of functions which cannot be computed directly, but only estimated via noisy observations.
Questions tagged [stochastic-approximation]
82 questions
12
votes
0 answers
Convergence of a Stochastic Process - Am I missing something obvious?
In the paper On the Convergence of Stochastic Iterative Dynamic Programming Algorithms (Jaakkola et al. 1994) the authors claim that a statement is "easy to show". Now I am starting to suspect that it isn't easy and they might have just wanted to…
Felix Benning
- 2,889
8
votes
1 answer
Variance converging to zero implies weak convergence to delta measure?
I have the following question:
Suppose that I have a sequence of random variables $X_n$ such that all moments exists and are finite. I have that $E[X_n]\to a$, where $a$ is a finite number and also $\text{Var}(X)\to 0$. We may also assume that a…
Liealgebrabach
- 622
7
votes
1 answer
Proof for convergence of stochastic gradient descent to a local optimum for non convex functions
Let's say I have a (multivariable) function $F(x) : \mathbb{R}^n \rightarrow \mathbb{R}$, which I would like to minimize.
There are no assumptions made on $F$, besides it being differentiable and having a gradient of the form
$$\nabla F(x) \propto…
Markus Peschl
- 387
- 1
- 3
- 16
7
votes
1 answer
Simple application of Donsker's theorem
I am trying to do exercise 5.15 in Moerter's book "Brownian Motion". It seems quite easy, but I can't solve it somehow: Suppose $S(j)_j$ is a SRW on the integers, started at zero. Show…
Liealgebrabach
- 622
5
votes
0 answers
Consequence of Dvoretzky Stochastic Approximation Theorem
I am having some problems trying to apply Dvoretzky Stochastic Approximation Theorem to one Lemma used in a paper I found about the proof of convergence of some reinforcement learning temporal difference methods.
Jaakkola & Jordan & Singh claim that…
Kareit
- 51
5
votes
1 answer
Algorithms for Stochastic Continuous Optimization
Question
I have a continuous optimization problem of the form $$ \max_{x \in \mathbb R^n} f(x), $$ where $f:\mathbb R^n \rightarrow \mathbb R$ is mostly smooth and bounded above. The standard approximation algorithms for these kinds of problems,…
Frank Seidl
- 1,016
5
votes
2 answers
How likely are two events to occur at the same time?
Let's think of two events $1$ and $2$.
Both events happen randomly $n_1$/$n_2$-times during a given time $T$ and last for a time of $t_1$/$t_2$.
What is probability $P$, that both events happen simultaneously at some moment?
EXAMPLE 1:
$T = 60$…
lukas.simon
- 77
5
votes
0 answers
Finding a unique strong solution
I am brushing up on my stochastic approximation. I am having a hard time with the following problem. I have the equation
$dX_t = \ln(1+ X_t^2)dt + X_tdB_t$
$X_0 = x$, with $x \in\mathbb R$
I know that this equation has a unique, strong solution. How…
Phil.D
- 51
4
votes
0 answers
Time Discretization
I wonder why we work with constant discretization in Time Discretization
of numerical approximation for numerical scheme if we take not necessarily constant Discretization Is the numerical scheme (Such : Euler, Runge-Kutta4, Euler-maruyama,…
Educ
- 4,940
4
votes
0 answers
Is there something like "stochastic induction"?
I'm trying to prove convergence of a stochastic approximation-like algorithm. I have two questions about prove-techniques when working with randomness.
1. For a non-random sequence $(a_t)_t$ one could prove $a_t \rightarrow a$ by induction as…
Nick Halden
- 118
4
votes
1 answer
Convergence of a real sequence (stochastic approximation)
Let $(\gamma_n)_{n \geq 1}$ be a sequence of positive real numbers satisfying $$\sum_{n \in \mathbb{N}}\gamma_n = + \infty \text{ and }\sum_{n \in \mathbb{N}}\gamma_n^2 < + \infty$$
I would like to know if the sequence $(u_n)_{n \geq 1}$ defined…
mfrt
- 138
4
votes
0 answers
difference of independent Rayleigh random variables
How do I find the probability density distribution (pdf) of the difference of independent Rayleigh random variables (whose probability density functions are known)? Assume $X$, $Y \sim \text{Rayleigh}(\sigma)$, $X$ and $Y$ are i.i.d. Then what…
Ricky
- 41
- 2
3
votes
0 answers
Maximize or find an upper bound of the function $kx^{k-1}\exp(-\mu(x^k-x))$
I was programming some random variable simulation using the acceptance-rejection method and I encounter with the Weibull$(k,\lambda)$ distribution. This random variable is posible to simulate with this method when $k \geq 1$. To do this I use a…
user111216
- 269
3
votes
0 answers
Application of Gram-Charlier expansion for Swaption pricing with drift extension
I am doing a project for university and I'm stuck at some point.
The aim is to implement the Gram-Charlier expansion into the 2-factor Hull-White model with Drift extension.
I already found out how to implement it without drift extension but I'm…
cleoser
- 31
3
votes
0 answers
Polynomial chaos expansion and ODEs
I originally asked this question on stats.SE but I didn't even get a handful of views. So I figured that here is probably a more appropriate site to ask. I am trying to figure out how to use PCE to quantify uncertainty in solutions of ODEs.
So if I…
rom
- 841