Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [objective-function]

23 questions
10
votes
1 answer

XGBoost custom objective for regression in R

I implemented a custom objective and metric for a xgboost regression. In order to see if I'm doing this correctly, I started with a quadratic loss. The implementation seems to work well, but I cannot reproduce the results from a standard…
Peter
  • 7,896
  • 5
  • 23
  • 50
10
votes
2 answers

Optimising for Brier objective function directly gives worse Brier score than optimising with custom objective - what does it tell me?

I am training an XGBoost model and as I care the most about resulting probabilities, not classification itself I have chosen Brier score as a metric for my model, so that probabilities would be well calibrated. I tuned my hyperparameters using…
Xaume
  • 212
  • 3
  • 14
7
votes
1 answer

What is a good objective function for allowing close to 0 predictions?

Let's say we want to predict the probability of rain. So just the binary case: rain or no rain. In many cases it makes sense to have this in the [5%, 95%] interval. And for many applications this will be enough. And it is actually desired to make…
Martin Thoma
  • 19,540
  • 36
  • 98
  • 170
4
votes
2 answers

Appropriate objective function and evaluation metric when I DO care about outliers?

I am reading these two pages: xgboost documentation Post on evaluation metrics I have a dataset where I am trying to predict future spend at the user level. A lot of our spend comes from large spenders, outliers. So, we care about them. I am using…
Doug Fir
  • 165
  • 1
  • 6
4
votes
1 answer

neural network function approximation with constraints

I would like to approximate a function $f(\cdot)$ by means of a neural network given a finite set of observations $f(x_i)$ where $x_i\in\mathbb{R}^n$ and $i=1\dots,N$. However, I have some prior knowledge on how this function should behave, for…
user149575
  • 41
  • 1
3
votes
2 answers

Optimizing MAE degrades MAE metrics

I have run a lighgbm regression model by optimizing on RMSE and measuring the performance on RMSE: model = LGBMRegressor(objective="regression", n_estimators=500, n_jobs=8) model.fit(X_train, y_train, eval_metric="rmse", eval_set=[(X_train,…
Mark531
  • 131
  • 3
2
votes
0 answers

Image reconstruction using low-light components

Let's say we have a regular photo and three low-light photos illuminated in different colors. Each pixel is a three-component vector $q=(R,G,B)$. Then $q_k^{A}$ is the $k$-th pixel of the regular photo and $q_k^{B}$ $q_k^{C}$ $q_k^{D}$ be the…
Don Draper
  • 141
  • 6
2
votes
1 answer

Non-linear Regression

For example suppose I've data set which looks like: [[x,y,z], [1,2,5], [2,3,8], [4,5,14]] It's easy to find the theta parameters from those tiny data set. Which is theta = [1,2,0] z = 1*x + 2*y + 0 But if my data set are non linear.…
Surya Bhusal
  • 143
  • 8
1
vote
1 answer

What is a good reward function when objective is to minimize the average along with the variance?

I am trying to formulate a problem where we are trying to minimize the average resource allocated to different users. Due to some inherent properties of the environment, some users can be easily minimized while it is difficult for other users due to…
user3656142
  • 181
  • 1
  • 6
1
vote
1 answer

XGB custom objective function - small change to default regression squared error objective function

Where can I find the code for the default squared error objective function? I just want to make a small change to re-weight certain datapoints?
xxanissrxx
  • 133
  • 6
1
vote
0 answers

Stochastic gradient descent (SGD)

The objective function () = [1∑=1Lossℎ(()⋅())]+2‖‖2 where Lossℎ()=max{0,1−} is the hinge loss function, ((),()) with for =1,… are the training examples, with ()∈{1,−1} being the label for the vector (). how to find the sgd with…
user3159445
  • 71
  • 2
1
vote
0 answers

Optimization problem with different type of constraints

I'm new to optimization problems. I want to find optimum values for my objective function. You can imagine my function as E = f(t1, t2, t3). I want to minimize E and following constraints limit the variables: 1- 0 < t1, t2, t3 < 255 2- t1, t2, and…
Babak.Abad
  • 121
  • 4
1
vote
0 answers

Deeplearning without an objective function?

In this article, the author talks about how deeplearning models no longer are trained for an objective function that humans specify, but find their own objective function. Specifically, he is talking about GANs. Is there a good resource explaining…
user637140
  • 201
  • 1
  • 5
1
vote
2 answers

How to determine the function is linear in linear regression problem?

I know that the first degree of the polynomial equation is considered as a linear function. But, I found some things confusing in linear regression. 1. f(x)= w1 x1+ w2 x2 + W3 x3 --> linear function 2. f(x)= w1 x1+ w2 x2 + W3 x1 x3 --> is it…
Animesh Kumar Paul
  • 111
  • 1
1
vote
0 answers

Linear regression space transformation

Can someone help me how space transformation works on linear regression problems because I have been confused. When we perform space transformation with a function e.g. $\varphi (x)$ we perform the transformation on dependent variable, independent…
Er1Hall
  • 11
  • 2
1
2