There are problems with the scoring system but what you suggest would mean playing for an hour and 10 minutes, for every game. That isn't what most people want. 4 ends seems to be the minimum for a "fair" game. You can get into an ebb and flow for a short time and set up strategies etc.
Also, more to the point of the scoring system. If I steal 1 in the first three ends, then my opponent scores 2 in the 4th. Me only being awarded with a 1 point win, isn't accurate. If the scoring system takes into account "ratings" then it should definitely take into account "stolen" points. It is much harder to steal a point, than to be held to a point with the hammer. To do it end after end, is much, much harder and should be viewed as such.
So, there are issues with the points and rating system but really, it's not like the game is falling apart.