Awesome enty

Thanks! I wrote a brief description under BeepBoop/Understanding_BeepBoop, but I'll release the code too once I get it cleaned up.

Me too, and I'll maybe revisit it at some point. Theoretically a deeper embedding model could learn feature interactions like "wall-ahead is more important when velocity is 8 than when it is 0"

I’m surprised as well. Btw, how many layers are you using in the deeper model? And is that fully connected? I guess some deeper models with explicit feature interactions may work better in robocode scenario, given high noise. I would try things like Deep&Cross, DeepFM, etc.

I tried a few (pretty simple) variants:

• Multiplying the features by a weight matrix. One nice feature of this is that a diagonal matrix recovers standard feature weighting, so this model should be strictly better than per-feature weights.
• A one-hidden-layer feedforward network.
• Summing up the embeddings from the above two.

I totally agree that allowing multiplicative feature interactions as you suggest should work better though!

One more detail, are you doing any encoding before inputting them into the NN part? I remembered Darkcanuck had some rather succesful attempt in NN (end-to-end), by binning features like the old VCS ways.

And since most features are essentially tabular, apart from the NN approaches with explicit feature interaction, GBDT can work very well as some feature transformation & interaction tool. There are also approaches using GBDT to do clustering, by converting clustering into classifying “dense” & “sparse” of space.

I'm using no special encoding, just normalizing the features so they are between 0 and 1. Decision-tree-like algorithms have been tried in robocode before (e.g. Wiki_Targeting/Dynamic_Segmentation), but not in conjunction with clustering/KNN as far as I know.

It's possible that the KNN already takes that into account sufficiently. Maybe if you bump the cluster size up a lot, and change the kernel width for cluster weighting, it might force this part of the learning into the NN instead?

It feels like some cascade model (widely used in ads & recommendation), by putting one deep model on top of some simple & very fast model, with the input of the former being the output of the latter. This architecture simplifies computation by magnitude of degrees, but also restricted the power of the deep model. A best practice then is to make the simple model fitting the output of the deep one, and retrain deep model on top of new input, and repeat...

I tend to think it's right that the KNN would take such relationships of features into account in a sense, but as a statistical model what it cannot do is generalize, which increases the number of data points needed to effectively cover some areas of the input space. In many ways, for this sort of usage, I would conceptualize the potential advantage of a deep embedding not as learning the feature interactions themselves, so much as learning the generalized contour of when to de-weight features, as a noise filter of sorts. This is a bit of a tangent, but thinking of in in terms of being like a noise filter, and also considering things like BeepBoop's velocity randomization, I also start to wonder if there could be some value in including not just the present feature values as inputs to deep embeddings, but including several ticks worth of feature history. Let the embedding learning have the potential to construct it's own temporally filtered (or rate of change) features.

Including several ticks of history seems like a nice way of removing the need for hand-crafted features like acceleration, time-since-velocity-change, distance-last-k-ticks, etc., and having the model learn them instead. Maybe a good model could even learn some PM-like behaviors.

Definitely a weakness of KNNs is generalization to new parts of the input space. I did think a bit about pre-training a model against a lot of bots and then quickly adapting it to the current opponent (maybe using meta-learning methods) so it would generalize better early in the match before it gets lots of data. On the other hand, aiming models get a lot of data pretty quickly, so I'm not sure of how much of an issue poor generalization really is.

I would say it probably depends what you're targeting. When targeting a strong surfer, I would say there's potentially a lot of value in maximizing the utility of data learned since the surfer last got information from collisions, and so that's a scenario where generalizing seems potentially more important in my eyes.

(unless it's going 100% flattener, in which case I would say the value is adapting on time scales that are simply different from what it's flattening over, either learning faster the flattener, or learning long-term "history repeating itself" trends/patterns that it loses sight of)

Also some deep enough model can learn how surfers (without flattening) "surf" hits, just like networks like "Deep Interest Network" used in CTR that learns how users' interest change over time. However our current use of KNN allows nothing like this. Maybe some end-to-end approach exists for robocode scenario.

My past experiments (shallow NN that does online-learning without pre-training) with end-to-end approach didn't yield anything interesting though.

Yeah, I also tried a purely NN gun and didn't get great results. You can train a NN offline to really high accuracy against random movement bots, but online it's much harder because data isn't iid.