In my opinion, the best way to do it would be to award points based on holding certain positions that put your opponent in chancery, like the double wrist lock, or front head lock, or leg ride, or crucifix, or ezekiel, or scarf hold, or saddle, and so on.
If they don't finish, then they're still ahead by however many points, and still performing in ways that more proximately lead to finishes anyways; and if they do finish, well then that's just gravy.
I would consider such a formulation a good training tool for preparing for a 'raw' sub only match as well.
One argument i've seen around points is that in some respects a point system can actually increase opportunities for submissions, in case the opponent(s) will be more focused on defending a scoring maneuver as well on top of just submission defense alone.
I personally think there's some logic to that depending on how it's implemented, so you might have holding certain chancery positions long enough count as a fall in of itself, even without a tap. I think the judo pinning rules are a good example/implementation of this, though i might expand it to include both pinning shoulders flat or chest flat.
If this ends up looking a lot like a catch wrestling ruleset, i don't think it's a coincidence.
Another thing to think about is looking 'outside the box', and consider rulesets or dynamics that extend beyond a single individual match isolated in space-time.
For example, what if in a folkstyle dual-meet, team victory was awarded to the side that got the most pins?
The character of coaching, training, and competition would change over night.
Quintet takes this approach in their events, and it has produced, each time in each event, and in only two events, some of the finest scenes of grappling action i've seen in a good long while.
