When you're new you improve fast, and you can gauge your rate of improvement by how much better you do over time rolling with higher belts. After a while (late blue or so for most) you know most of the main positions and that exponential improvement slows down. Now it's harder to gauge using rolling because most of the higher belts are improving at about that same rate. At this point I think you can gauge yourself one of two ways: either by tournament performance or by testing your ability to add skills. The former is obvious (if you were taking silvers and bronzes or not medalling and now you're consistently winning your division, you're getting better), the latter requires a little more explanation. After you've learned most of the main positions and techniques, you start specializing your game. Most purple belts and up have a reasonably well defined A game that works against other advanced guys. Deviating from that game will often get you in trouble against people your own level. As you work on specific positions and learn them deeply, you should be able to use them in rolling and competition successfully. If you're consistently adding skills in that manner, you're getting better.
There's an Eddie Cummings interview where he talks about Garry Tonon's development and says something along the lines of 'Garry used to have only a RNC, now he has a high elbow guillotine and a leg lock game'. It's not as if Tonon didn't know guillotines or footlocks 3 years ago, but he couldn't apply them against high level people with consistent success. Now he can, so now he 'has' those moves as part of his game. That's probably the best way for someone who is already pretty good to measure progress, in terms of diversifying what could reasonably be called your A game.