Particle Swarm Optimization

PSO is a fairly well-established method of stochastic optimization so there are many references. The foundational work was:

Eberhart, R. C. and Kennedy, J. A new optimizer using particle swarm theory. Proceedings of the sixth international symposium on micro machine and human science pp. 39-43. IEEE service center, Piscataway, NJ, Nagoya, Japan, 1995.

In a nutshell: Solution points are given a velocity in the solution space. The velocity is modified towards the global best and the "personal" best for that solution. The set of solutions then moves through the solution space with velocities updated at each iteration.

Notes:

The actual velocity update is randomized as follows:
v = v + c1 * rand() * (pbest-present) + c2 * rand() * gbest-present)
where c1 and c2 are "constants" (they may change from one iteration to the next, but they are the same for all particles in an iteration), rand() is a random variable, pbest is the position of the best solution for this particular particle and gbest is the position of the best solution found globally.

c1=c2=2.0 is popular.

In order to keep the particles from leaving the valid solution space, clamping is usually applied on both the velocity within each dimension and the resulting location.

Since the global best is kept for purposes of velocity, this is effectively an elitist algorithm.

There are no genetic operators (crossover or mutation).

This is a heuristic - convergence is highly likely, but not guaranteed. There is no good way to derive how close a solution is to the actual global maximum based on the algorithm performance. There may, of course, be other things known about the problem that can provide an upper bound for the difference.

The main upside of the algorithm is its simplicity. There are few tuning parameters and practically no restrictions on the fitness function (other than it being computable). Convergence is generally quite fast.

The downside is that the algorithm is very prone to getting stuck on local maximums, particularly if the gradient of the fitness function steep in the vicinity of the local extreme point. Once the particles have been pulled away from a global maximum due to faster convergence at a local maximum, there is nothing that pulls them back that direction.