Questions tagged [pac-learning]
19 questions
40
votes
5 answers
Are decision tree algorithms linear or nonlinear
Recently a friend of mine was asked whether decision tree algorithms are linear or nonlinear algorithms in an interview. I tried to look for answers to this question but couldn't find any satisfactory explanation. Can anyone answer and explain the…
user2966197
- 541
- 1
- 6
- 8
4
votes
1 answer
A trick used in Rademacher complexity related Theorem
I am currently working on the proof of Theorem 3.1 in the book "Foundations of Machine Learning" (page 35, First edition), and there is a key trick used in the proof (equation 3.10 and 3.11):
$$\begin{align*}
&E_{S,S'}\left[\underset{g \in…
learning machine
- 161
- 6
4
votes
4 answers
Where does the "deep learning needs big data" rule come from
When reading about deep learning I often come across the rule that deep learning is only effective when you have large amounts of data at your disposal. These statements are generally accompanied by a figure such as this:
The example (taken from…
Aran
- 41
- 1
3
votes
1 answer
What is PAC learning?
I have seen here but I really cannot realize that.
In this framework, the learner receives samples and must select a generalization function (called the hypothesis) from a certain class of possible functions. The goal is that, with high probability…
Green Falcon
- 14,308
- 10
- 59
- 98
3
votes
1 answer
Generalization Error Definition
I was reading about PAC framework and faced the definition of Generalization Error. The book defined it as:
Given a hypothesis h ∈ H, a target concept c ∈ C, and an underlying distribution
D, the generalization error or risk of h is defined by…
Green Falcon
- 14,308
- 10
- 59
- 98
2
votes
1 answer
Why does PAC learning focus on learnability of the hypothesis class and not the target function?
The definition of PAC learning is roughly
An algorithm is a PAC learning algorithm if it given enough data, for any target function, it asymptotically does as well as it possibly could given the functions it's capable of representing.
This…
Jack M
- 285
- 1
- 6
2
votes
1 answer
Why is a lower bound necessary in proofs of VC-dimensions for various examples of hypotheses?
In the book "Foundations of Machine Learning" there are examples of proving the VC dimensions for various hypotheses, e.g., for axis-aligned rectangles, convex polygons, sine functions, hyperplanes, etc.
All proofs first derive a lower bound, and…
learning machine
- 161
- 6
2
votes
1 answer
Generalization bound (single hypothesis) in "Foundations of Machine Learning"
I have a question about Corollary $2.2$: Generalization bound--single hypothesis in the book "Foundations of Machine Learning" Mohri et al. $2012$.
Equation $2.17$ seems to only hold when $\hat{R}_S(h)
learning machine
- 161
- 6
2
votes
1 answer
A question on realizable sample complexity
I came across the following exercise, and I just can't seem to crack it:
Let $l$ be some loss function such that $l \leq 1$. Let $H$ be some hypothesis class, and let $A$ be a learning algorithm. show that:
$m^{\text{stat, r}}_H (\epsilon) =…
Nadav Schweiger
- 21
- 1
2
votes
1 answer
PAC Learnability - Notation
The following is from Understanding Machine Learning: Theory to Algorithm textbook:
Definition of PAC Learnability: A hypothesis class $\mathcal H$ is PAC learnable
if there exist a function $m_H : (0, 1)^2 \rightarrow \mathbb{N}$ and a learning…
tkj80
- 139
- 3
2
votes
1 answer
Meaning of Instance Space and Concept Class, (PAC Learnable)
I'm studying Probably approximately correct learning, and I don't understand what an Instance Space and a Concept is.
I have see that wikipedia https://en.wikipedia.org/wiki/Probably_approximately_correct_learning provides various examples, but…
Tommaso Bendinelli
- 275
- 1
- 9
2
votes
0 answers
Intuition behind Occam's Learner Algorithm using VC-Dimension
So I'm learning about Occam's Learning algorithm and PAC-Learning where for a given hypothesis space $H$, if we want to have a model/hypothesis $h$ that has an True error of $error_D \leq \epsilon$, with a probability of $(1-\delta)$ for a given…
Joe
- 101
- 7
1
vote
0 answers
Uniform convergence garantee on sample complexity
I can't understand why the Uniform Convergence guarantees an upper bound and not a lower bound on sample complexity as stated on [1]
Corollary 4.4.
If a class $H$ has the uniform convergence property with a function $m^{UC}_H$
then the class is…
Enrico R.
- 11
- 1
1
vote
1 answer
Disproving or proving claim that if VCdim is "n" then it is possible that a set of smaller size is not shattered
Today in the lecture the lecturer said something I found peculiar, and I felt very inconvenient when I heard it:
He claimed, that if the maximal VCdim of some hypothesis class is $n\in\mathbb N$, then it is possible that there is some $i
C.H.
- 11
- 2
0
votes
1 answer
VC-dimension proof for a family of classifiers
I've been working on determining the VC-dimension of a specific family of classifiers, and I would like to get some feedback on the proof I've come up with. The family of classifiers is defined as follows:
$$
\mathcal{H} = \{h_{a,b} : a, b \in…
Estiven
- 1