A Turing machine is just one of many ways in which you could make a computing device. It is one of the easiest to mechanize and program for, which is why we use them but there is absolutely no reason to believe it is the only way of doing things. GP was getting at a more basic form of computation than the specific one performed by Turing machines. From a mathematical perspective the tape, the registers and instruction tables are all implementation details.
> absolutely no reason to believe it is the only way of doing things
Well, I did not suggests that.
> From a mathematical perspective the tape, the registers and instruction tables are all implementation details.
No, they are used to define concepts such as space complexity for Turing machines and there is something to learn from Turing machines, that's why people are studying them.
A function can be seen both as the set of all pairs of the function, or as an oracle that you feed input to and receive output. The first is to me the full memory space, the second is maybe computation.