LRP (ish)

Jump to navigation Jump to search
Revision as of 5 December 2014 at 19:21.
The highlighted comment was created in this revision.

One thing I really missed from the old rumble was the LRP, but without ELO/Glicko we can't really do the whole straight-line fit any more. So, instead I have added a Score Distribution image on every bot's details page. The red is APS and the green is Survival (as seen in image the mouseover). The image is directly embedded in the HTML using data URIs, so if you are using IE, 8 and later only, otherwise pretty much everything supports it. I'm also planning to add this to the BotCompare page so you can analyse differences in score compared to opponent score for both APS and survival.

    Skilgannon (talk)23:28, 10 May 2013

    Ahhh, neat stuff. That's very nifty with directly embedding the image data there. For some reason the image is displaying very tiny for me though under Firefox 20.0. It gets scaled to the box around it properly under Chromium, but not Firefox.

    EDIT: Nevermind... the styles.css file was being cached and that was the problem. A ctrl-r fixed it.

      Rednaxela (talk)23:48, 10 May 2013

      Ah yeah, the styles.css was changed so you need to do a hard-reload.

      I've now added the KNNPBI to the bot-details Scores Distribution, and the bot-compare has a Diff Distribution.

        Skilgannon (talk)13:50, 11 May 2013
         

        There is something fishy with a chart in the right part close to the end. If you look at above CunobelinDC score distribution you would see that there is no corresponding red points for stronger opponents, while blue and grean are there. This is quite common theme for other bots as well.

        Also have a look at this EvBot score distribution you would see the problem with normalizing, i.e. about 1/4 of the space in the right part of the chart has no points. Which is non optimal use of the chart space.

          Beaming (talk)16:44, 18 November 2013

          Is it still showing the problem? I don't see anything wrong right now. I had some issues with (I suspect) bad bytecode and versioning, but that should be fixed now.

          As for the EvBot chart, that is because in meleerumble nobody gets higher than ~75%, so the top 25% is empty. Although I guess I could normalise to the top score, I'd rather have the charts consistent as better bots are released.

            Skilgannon (talk)17:55, 18 November 2013

            Aha, I see now why melee charts were somewhat off.

            But I insist that I do not see red points for X>95% for CunobelinDC. Look at 5 the rightmost green points, I cannot locate red (APS) or blue for the same X values. It might be aliasing problem or may be points are just on top of each other.

              Beaming (talk)18:41, 18 November 2013

              Green is survival, and so the X value is the average survival score of the enemy bot. The red and blue use enemy APS as the X value, not survival, and since survival scores are higher the green dots go further to the right.

              I've actually thought about changing the X axis to just be enemy APS to make it easier to interpret. Or ordering the X-axis by rank instead of using APS values.

                Skilgannon (talk)09:22, 19 November 2013

                I've changed it so they all use APS on the X axis, so it should be clearer now.

                  Skilgannon (talk)11:03, 19 November 2013

                  I like new charts better and they are easier to understand. Thanks.

                  They are some white space on left and right, but I think it is ok since presentation is clearer now.

                  There is amasing correlation between APS and Survival in roborumble scores :)

                    Beaming (talk)14:51, 19 November 2013
                     

                    Would it be a good idea, to make survival difference green on bot comparison charts? This way color scheme matches the main chart.

                      Beaming (talk)15:26, 19 November 2013

                      Done.

                        Skilgannon (talk)22:23, 19 November 2013

                        Wow. I did not expect such quick fix. Thanks Skilgannon.

                        Now I would like to complain about this green color :) I personally think it is not very contrast on a white background.

                        I like our red and blue, but I think it would be better to replace currently light colored green with either much darker green or better yet something else, for example black.

                        I only speak for my own eyes, but I think our potentially color blinded participants will appreciate it as well.

                          Beaming (talk)23:14, 19 November 2013

                          Wow, sorry I missed these. The reason I use this green is because I just do a simple RGB 3-layer image, and set the different pixels manually. I would have to do a much more complex transform to get better colours. If you want to investigate my uber-messy code and provide a patch feel free =)

                            Skilgannon (talk)19:23, 18 November 2014

                            Hi Skilgannon,

                            I am not a plotting wizard but I plot enough in my life to make a fullish attempt to improve it a bit.

                            Though, it looks like the colors were chosen for the speed and not beauty, and it might be hard to have still fast yet beautiful output.

                            But I'll try to help. How can I look at the code?

                              Beaming (talk)03:12, 25 November 2014

                              Check here =)

                                Skilgannon (talk)10:48, 25 November 2014

                                Here is the suggested patch. Note that image array now created in uint8 RGB space, since resulting png image is in RGB space as well. As result it removes unnecessary image manipulation, after the array is prepared.

                                Beware, I was not able to fully test it since I am missing database and google modules. But it should work fine.

                                Note that colorAPSvsKNPBI is not used in this script but probably needed for another python script responsible for a given bot stats.

                                --- BotCompare.py	2014-09-15 15:40:19.000000000 -0400
                                +++ BotCompare.new.py	2014-11-26 22:28:50.000000000 -0500
                                @@ -266,28 +266,34 @@
                                                 
                                                 out.append("</td><td rowspan=\"7\">")
                                                 enemyScores = pickle.loads(zlib.decompress(rumble.ParticipantsScores))
                                +                # RGB color model
                                +                # Default colors
                                +                # colorSurvival = (0,255,0)
                                +                # colorAPS = (255,0,0)
                                +                # colorAPSvsKNPBI = (0,0,255)
                                +                # Colors suggested at 
                                +                # http://ksrowell.com/blog-visualizing-data/2012/02/02/optimal-colors-for-graphs/
                                +                colorSurvival = (62,150,81)
                                +                colorAPS = (204,37,41)
                                +                colorAPSvsKNPBI = (57,106,177)
                                                 size = 169
                                -                a = numpy.empty((size+1,size+1,4), dtype=numpy.float32)
                                +                a = numpy.empty((size+1,size+1,3), dtype=numpy.uint8)
                                                 a[...,(0,1,2)]=255
                                                 for cp in commonList:
                                                     eScore = enemyScores.get(cp.Name,None)
                                                     if eScore:
                                -                        a[max(0,min(size,size-int(round((cp.A_APS - cp.B_APS + 50)*0.01*size)))),
                                -                          int(round(eScore.APS*0.01*size)),(0)]=0
                                +                        x = int(round(eScore.APS*0.01*size))
                                +                        y = max(0,min(size,size-int(round((cp.A_APS - cp.B_APS + 50)*0.01*size))))
                                +                        a[x,y,(0,1,2)] = colorAPS
                                                         
                                -                        a[max(0,min(size,size-int(round((cp.A_Survival - cp.B_Survival + 50)*0.01*size)))),
                                -                          int(round(eScore.APS*0.01*size)),(1)]=0
                                +                        y = [max(0,min(size,size-int(round((cp.A_Survival - cp.B_Survival + 50)*0.01*size))))
                                +                        a[x,y,(0,1,2)] = colorSurvival
                                 #                        if eScore.ANPP > 0 and b.NPP >= 0:
                                  #                           a[size-int(round(b.NPP*0.01*size)),int(round(eScore.ANPP*0.01*size)),(0,1)]=0
                                                 
                                -                b = Image.fromarray(a.astype("uint8"), "CMYK")
                                -                c = cStringIO.StringIO()
                                -                b = b.convert("RGB")
                                -                a = numpy.array(b)
                                -                a[(a == (0,0,0)).all(axis=2)] = (255,255,255)
                                                 a[size - int(round(.5*size)),...] = 127
                                -                a
                                -                b = Image.fromarray(a,"RGB")
                                +                b = Image.fromarray(a.astype("uint8"),"RGB")
                                +                c = cStringIO.StringIO()
                                                 b.save(c,format="png")
                                                 d = c.getvalue()
                                                 c.close()
                                
                                
                                  Beaming (talk)04:37, 27 November 2014
                                   

                                  I'm very busy at the moment, but I'll take a look at this on the weekend.

                                    Skilgannon (talk)08:03, 3 December 2014

                                    No problem. I also made a pull request at the bitbucket to simplify import. This request is completely analogous to the presented here patch.

                                      Beaming (talk)23:42, 3 December 2014

                                      I've integrated your changes, with a few modifications of my own (just implementation details - averaging colours for multi-hit pixels). I also added it to the BotDetails page.

                                        Skilgannon (talk)20:21, 5 December 2014
                                         
                                         
                                         
                                         
                                         
                                         
                                         

                                        Sorry to be such a pain. But I have one more thought. May be APS as X axis is not such a good choice. Since in meleerumble there is white spaces on left and right. No bots have APS above 70% and below 10%.

                                        Would it be better to use rating? Or may be ANPP? Those take full scale.

                                          Beaming (talk)03:00, 20 November 2013

                                          I use APS because it is stable and always available. ANPP is sometimes not available (just after stabilising, before batches have run). I could use rating, but that I feel is too closely fit to which other bots are in the rumble at the time of viewing, and I would prefer the image to be more stable for long-term comparison.

                                            Skilgannon (talk)19:26, 18 November 2014