The shift was a response to really good hitters and reduced action and risk. Then pitchers got REALLY good and made it even more of an all or nothing proposition to the point that college and prep private coaches weren't teaching contact hitting and small ball. It was bad for the sport and bad for fans.
I would say the shift was the response to the all or nothing approach to hitting. Once the teams noticed that hitters were not taking a two strike approach and going the other way, the shifted the defense in response.
The shift has been around (in small doses) back to Ted Williams. Dead pull hitters like Willie Stargell and Willie McCovey faced shifts frequently back in the day (mostly with no runners on base). Like basketball, analytics of hitting made defenses react accordingly. I wanted hitters to react (as you stated) to more contact and opposite field hitting on their own.
Oh well. Maybe because I was a weak hitting/ opposite field hitter through most of college (until I got stronger) may be coloring my thoughts a bit.