The biggest issue with assetbundles for models is the constant breaking of shaders embedded in the bundles as Unity gets updated, resulting in errors (skippable mainly) on decompressing bundles.
The worst of all is the recent bug (reported) where metal support on iOS is broken to the point of crashing an app if the assetbundle shaders are not correct (not sure of exact why).
I hope that in future asset bundles are not just able to be decompressed without errors but also shaders can be stored separately to the bundles or stored in a non-compiled form that prevents any of these problems after every major Unity update that can mean 100s of bundles need updating.
I would have thought materials would often be better stored in their own shared library anyway and referenced from bundles, to allow separate updating and less download (this is what we do, but still have problems with dummy shaders in bundles!).
But any work arounds would be good to know!