Modem Freezing? Dropping internet. topic

I've got a N5 from the play store right when it came out. I've noticed that randomly during the day if I'm using the internet via browser, apps, hangouts, etc the page / app will suddenly lose connectivity. Not just the app though, nothing on my phone can access internet. The status bar with show full bars, and 3G or H connectivity, and when I pull down the quick settings the connection is white, not red. So I toggle airplane mode, but when I do so there is a long delay where nothing happens. Maybe 20 or 30 seconds before airplane mode finally kicks in. Then after disabling airplane mode, I can use the internet again for a while before I have to repeat.

Any thoughts what might cause this? I've flashed older radios, wiped and flashed several Roms including full stock not rooted, and I keep having this issue.

There's a chance it could be my carrier, but my wife's phone on the same carrier (but a store-bought N5) never has this issue...

Thanks for your input!