You can't buy a GPU to train neural nets currently because of bitcoin mining etc. You can use Continuous Gray Code Optimization on clusters of CPUs. Each CPU has the full neural model and part of the training set (which can be local and private to the CPU.) Each CPU is sent the same short sparse list of mutations and returns the cost for its part of the training set. The costs are summed and if an improvement an accept message is sent to each CPU else a reject message. The mutation operator is random plus or minus a.exp(-p.rnd()). If the weights are constrained between -1 and 1 then a=2 to match the interval. The so called precision p is a problem dependent positive number. rnd() returns a uniform random between 0 and 1. Very little data is moving around per second. The CPUs can be anywhere on the internet really with a little engineering.
Пікірлер: 2