Yeah, I agree. If Ultron had actually been portrayed as Ultron, then he could have been an excellent villain. Even the cartoon Avengers show has had some really awesome Ultron episodes. I would have loved to see a big screen version of that, with the real Ultron. Unfortunately, they were more interested in being funny than telling an accurate or compelling story.
I was riding the Whedon hype train pretty hard after the first Avengers because I liked it so much, but Age of Ultron was the polar opposite of that. As much as I loved Avengers, I hated Age of Ultron. So I couldn't be more happy that he's gone from the MCU. Though, the after effects of Whedon are still felt to this day (with all the Whedon-esque shitty ass jokes, comedy, and one liners they force into each and every damn movie).
Some of the comedy in IW actually was quite funny. But a lot of the other stuff in the other films in phase 2 and 3 has been downright dreadful.