That sums up the loader issue quite nicely.This breaks a fundamental rule that was decided back in FFF-128 when Loaders were being considered. This rule is that there is only one way to move items between structures, by using Inserters. Even though the items were never actually transferred, and our intent was that the Loader consumes them, it is enough that the player might think they were transferred.
If you think about it, what is a "loader" to start with? It's effectively just a specialization of an inserter, with the added requirement of terminating a belt in order to achieve full throughput.
That "belt terminal" part of the loader is the only missing fragment there. No direct insertion capabilities or anything, just an "end of belt" or "begin of belt" terminator which has barely enough internal buffer capacity to fully scale with inserter stack size.
We have mods for "belt boxes", which explored belt integrated storage but ignored the "terminal" constraint (and all have visual issues, due to not actually integrating a belt), and we have mods exploring the original "loader" design, as an inserter-free solution. Both approaches have bypassed the "only inserters transfer to different inventory type" constraint though.
If you were to revisit the idea of a loader, as merely the combination of regular, vanilla inserter + belt terminal piece, it should be fitting the established constraints quite well. Visualization wise, a variation of the "underground belt entrance" with a "blocky / chest like" instead of a "curved" cover actually feels like a natural choice for such a belt terminal.