Difference between revisions of "Thread:Talk:ScalarBot/Version History/how to build a good test bed?"
m (New thread: how to build a good test bed?) |
m |
||
| Line 5: | Line 5: | ||
My test bed is running at 35 battles for 30 seasons with 10 bots — 300 battles in total, but it shows irrelevant with rumble score. Is that the bots I choose make it a bad test bed, or just because I have too little battles? | My test bed is running at 35 battles for 30 seasons with 10 bots — 300 battles in total, but it shows irrelevant with rumble score. Is that the bots I choose make it a bad test bed, or just because I have too little battles? | ||
| − | Again, it seems that even after ~3000 battles, the rumble score is still not reliable | + | Again, it seems that even after ~3000 battles, the rumble score is still not reliable to be used to compare two versions. |
Revision as of 03:04, 27 September 2017
Recently I tried a lot to tune the movement, and the result is promising — it performs very well in my test bed (which consisits of some bots I’m performing bad in the past, including some guess factor targeting bots, dc bots and a simple targeter with VG. However, the rumble result shows a huge performance regression ;/
Then, I tried another one, when published to the rumble, it shows huge performance increase (and a little increase when full pairing) — but after ~5000 battles, the performance is even decreased, comparing to the baseline version.
My test bed is running at 35 battles for 30 seasons with 10 bots — 300 battles in total, but it shows irrelevant with rumble score. Is that the bots I choose make it a bad test bed, or just because I have too little battles?
Again, it seems that even after ~3000 battles, the rumble score is still not reliable to be used to compare two versions.
