> If this is the case, it's just a matter of time until costs can be reduced.
Is it, though? We cannot predict technological advancement, and the times of ̶M̶u̶r̶p̶h̶y̶'̶s̶ ̶L̶a̶w̶ Moore's Law* for computational power are long gone. There is simply no guarantee that the costs will go down enough.
I think there is plenty of room to make AI inference much more energy efficient. For example, there are companies testing creating custom silicon to run the model. Once that technology matures and we have some "good enough" models for normal use, inference cost for non-bleeding-edge models can come way down.
I don't expect bleeding-edge models to become any cheaper, but previous generation models can potentially be really cheap.
Is it, though? We cannot predict technological advancement, and the times of ̶M̶u̶r̶p̶h̶y̶'̶s̶ ̶L̶a̶w̶ Moore's Law* for computational power are long gone. There is simply no guarantee that the costs will go down enough.
* thanks lucianbr!