A question on determinism
In my experiments so far, which have involved Python and P5.js (built on top of Javascript), I have been unable to obtain a single response/completion from the same prompt and parameter settings with...
View ArticleA question on determinism
There’s inherent non determinism in GPU calculations around floating point operations - the differences in log probabilities are tiny, but when there’s a small difference between the top two likely...
View ArticleA question on determinism
There are speed tradeoffs, and in order to make the endpoints fast GPUs are used, which do parallel (non deterministic) calculations. Any modern gpu neural net calculations will be subject to these....
View ArticleA question on determinism
Interesting topic. I was thinking through practical use-cases of this technology in a corporate setting and the stochastic nature was playing on my mind. I can foresee trying to explain this to sales...
View ArticleA question on determinism
Yes, we gotta accept the stochastic nature of the neural nets. It’s as if there was a new (human) employee in the enterprise. Even here, in this forum, I see a lot of people craving deterministic...
View ArticleA question on determinism
Thanks for all of these excellent comments. It is not so much a question of the value of determinism, but instead the attempt to characterize the uncertainty given that we have a stochastic process....
View ArticleA question on determinism
On the same GPU? Or between different GPU families/drivers/…? I have been able to repeat GPU calculations with repeatable results on the same GPU. Does this maybe imply that the server infrastructure...
View ArticleA question on determinism
H100 compute clusters are not mix and match devices, there could be H100 and A100 blocks and you could get either one, but not a mix of the two at the same time, I don’t see a coherent driver stack...
View Article
