Collections aren't strictly reference types and with ref structs in generics feature you will be able to write a LINQ-style abstraction on top of Span<T> (though lifetime restriction still apply).
(you could already do so provided it specialized on accepting spans only with element type being a generic argument, with a new feature you could have Span<T> implement e.g. IList<T> and be passed to e.g. Reduce<T, U> where T : Span<U>, and regular struct generics existed for a very long time)
All standard BCL collections (List, Dictionary etc) are reference types. Some associated machinery like enumerators are value types, but that's beside the point here, which is that by default you end up with shared data rather than a (safer) copy. This also means that whenever you return an owned collection you either have to manually copy or wrap it in a readonly proxy to avoid accidental mutation of internal state by the caller.
In contrast, in Swift, Array, Dictionary, and Set are value types, so if you return a collection you own from a function, the caller semantically gets a copy - under the hood they do copy-on-write, so until either side decides to change its collection, they continue to share storage. Similarly for parameters neither the caller nor the callee has to worry about accidentally mutating something someone else owns. And in the rare case where pass-by-ref and its shared mutability is actually desirable, you handle that with `inout` (which is mostly equivalent to C# `ref`, including explicit marker at call site).
The same goes for strings, which in turn allows strings to be mutable like any other collection (since they aren't shared all over the place).
The CLR type system allows for all the same things in principle, yes. And if C# and the BCL were designed from scratch today, I suspect that it would make very different choices. But these are some very fundamental things that were decided 20-25 years ago, and now cannot be changed for back-compat reasons.
(you could already do so provided it specialized on accepting spans only with element type being a generic argument, with a new feature you could have Span<T> implement e.g. IList<T> and be passed to e.g. Reduce<T, U> where T : Span<U>, and regular struct generics existed for a very long time)