Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [metadata]

18 questions
23
votes
5 answers

How to annotate text documents with meta-data?

Having a lot of text documents (in natural language, unstructured), what are the possible ways of annotating them with some semantic meta-data? For example, consider a short document: I saw the company's manager last day. To be able to extract…
Amir Ali Akbari
  • 1,393
  • 3
  • 13
  • 25
6
votes
0 answers

Meta-analysis of public 16S data

I am trying to start a meta-analysis for which I want to extract some 16S-based information from public databases. Moreover, I want to relate this information with any metadata found in the associated studies (everything from environmental variables…
André Soares
  • 161
  • 2
4
votes
2 answers

Recommendations for storing time series data

As part of my thesis I've done some experiments that have resulted in a reasonable amount of time-series data (motion-capture + eye movements). I have a way of storing and organizing all of this data, but it's made me wonder whether there are best…
lmjohns3
  • 588
  • 6
  • 19
3
votes
1 answer

Can metadata be used to adapt parsing for an unescaped in field use of the delimiter?

I have data coming from a source system that is pipe delimited. Pipe was selected over comma since it was believed no pipes appeared in field, while it was known that commas do occur. After ingesting this data into Hive however it has been…
Chris Simokat
  • 131
  • 3
2
votes
1 answer

Automatically uses several cores on R

I am using a library called MFE to generate meta-features. However, I am working right now with several files and I notice that I am using only 1 core of my machine and taking too much time. I have been trying to implement some libraries as I saw…
Guilherme Felipe Reis
  • 133
  • 6
1
vote
0 answers

Questions about adding metadata to a CNN using keras

I have a convolutional neural network and would like to include some metadata. My metadata is in a multiple csv files that correspond to each class and it contains a bunch of geometric properties (about 8 numerical measurements), specifically…
cdr
  • 33
  • 4
1
vote
0 answers

Error "comparison (6) is possible only for atomic and list types" encountered in metafor

I'm very new to r and trying to run a multi-level meta analysis using pre-calculated effect sizes. The data file can be accessed via this link:testrunfile The script I used as a first step to fit the model was: res <- rma.mv (yi = es_r, v = var,…
Jess D
  • 11
  • 3
1
vote
1 answer

What is meta- data and meta features?

I want to know what is metadata and what is meant by meta features? When I google Meta Features what I get is feature selection tool called "Meta-Feature". What is the function of feature selection tools ? Also, what I want is the definition and…
asmgx
  • 549
  • 2
  • 18
1
vote
0 answers

Labeling data as having an error?

I am curating a large quantity of data from different sensors. If I know that a particular sensor was broken or poorly calibrated for a particular time range, what would be a useful way of annotating the data to make it clear that the data are of…
David LeBauer
  • 111
  • 6
1
vote
1 answer

Understanding a dataset (prior to applying ML models) with no metadata given

How do you understand a dataset when there is no metadata given (no details about the attributes given in the dataset)? It is difficult to comprehend the attribute names as only the short forms are given. It's given to me that 'pm2.5' is the…
Aditya Kadrekar
  • 41
  • 1
  • 4
1
vote
0 answers

Antisymmetry of graph of Information vs probability

The formula for Information given by a data of occurring with probability p is: I=-log2 p This formula gives the bits if information needed to know the outcome of the event. This formula captures the intuition that the information needed to know…
user1825567
  • 1,416
  • 1
  • 14
  • 24
1
vote
0 answers

Reduce MNIST dataset

I am working on the MNIST dataset. How I can reduce 50% of this data? (x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
Salman Al-haddad
  • 11
  • 2
0
votes
1 answer

Regression on multiple datasets with a per-dataset variable

I have 10 datasets, each with the same variables (e.g., age and income) but different numbers of observations. Let us now consider a categorical variable $X$ that can only take values $0$ and $1$ per dataset, meaning that it keeps the same value for…
Pippo
  • 143
  • 2
0
votes
1 answer

How to work with input which is a combination of metadata+ vectorized text data + image pixel data to build a Regression Model (predict views)?

There are 4 datasets (all in csv format), each has a uniqueID column by which each record can be identified. Image and text datasets are dense datasets.(need to be converted to ndarray). Can someone suggest how to use all these 4 datasets for…
Mathew
  • 31
  • 3
0
votes
1 answer

Finding an appropriate price using thousands of data points

If i have a lot of data points describing the price of a used car. How would I find the market value of the car (assuming that the price points in the data set are the only determinant used, and the basis of determination will be the frequency…
Zee
  • 3
  • 2
1
2