Physical literacy is both general and sport specific, for sure. One issue I run into is that people often expect something like "instant success" in whatever they are doing. So being able to jump, crawl, hop (one leg, two legs, bunny, etc.), roll, skip, etc are fundamentals of human movement. Then there are more specialized, compound movements of a specific sport, Judo in this case. Which are made up of more specific versions of the fundamentals of human movement (physical literacy). So a guy can do O Goshi with uke walking right at him, and so thinks he has mastered O Goshi. I see this pretty often even when teaching competitive teenagers, although with time they start to get the point you made in your post. The various sutemi waza come to mind as a good example.