The gray should be fine for now - you are better off with clothes that fit and you are comfortable in than in something that would make you slip around the saddle more. I would consider upgrading to beige pants agian later on, but for now, I would wear the gray if they fit better. There are no set rules as to what colors shouldn't be worn (other than white breeches - those are pretty much reserved for dressage), as long as the clothes are neat and workmanlike, and create a clean image. I have never heard of any great horseman (George Morris for example) saying anything bad about the "colors" of a rider's breeches - are you clean, do your clothes fit, is your horse clean and in good health, and are you both turned out as good as possible (i.e. boots wiped off, horse's mane braided, pads clean and neat, etc). Those things are much more important!
Of course, in flat classes judges may have a preferance for the typical beige breeches and navy jackets, but here agian, they are only preferances and shouldn't have anything to do with how you are judged. Since you are jumping in a jumper division, there is a lot more room to play with as far as "fashion" goes.