I have a 3D spaceship duel simulation (without the source code).
I need to build a model that will learn the simulation's behavior.
I can run the simulation as many times as I want, randomly feeding it inputs (velocities, ranges etc) and the simulation output is whether my spaceship won or didn't.
I've used the simulation to create 3 million samples and tried to train a model using the dataset via deep learning.
I can't seem to get accuracy higher than 0.88.
So far I've tried using MLP and tuning the epochs, batch, size and depth of the net, adding features and adding derivative features (trigonometric etc).
Another thing I tried is splitting the dataset into 2 sets of data (logically partitioning to 2 scenarios) and built 2 models to solve each set independently.
Each model predicts it's own set well (~ 0.96) but combining the 2 models doesn't work well.
These scenarios are linked to the winning or losing (the initial label I'm trying to learn) so I can't make a model to choose the scenario and then use the relevant model.
Iv'e tried several things (including stacking, using decision tree to try and predict the scenario etc)
I would appreciate any leads or advice.
