Particle Swarm Optimization

PSO is a fairly well-established method of stochastic optimization so there are many references. The foundational work was:

Eberhart, R. C. and Kennedy, J. A new optimizer using particle swarm theory. Proceedings of the sixth international symposium on micro machine and human science pp. 39-43. IEEE service center, Piscataway, NJ, Nagoya, Japan, 1995.

In a nutshell: Solution points are given a velocity in the solution space. The velocity is modified towards the global best and the "personal" best for that solution. The set of solutions then moves through the solution space with velocities updated at each iteration.

Notes:

The actual velocity update is randomized as follows:
v = v + c1 * rand() * (pbest-present) + c2 * rand() * gbest-present)
where c1 and c2 are "constants" (they may change from one iteration to the next, but they are the same for all particles in an iteration), rand() is a random variable, pbest is the position of the best solution for this particular particle and gbest is the position of the best solution found globally.

c1=c2=2.0 is popular.

In order to keep the particles from leaving the valid solution space, clamping is usually applied on both the velocity within each dimension and the resulting location.

Since the global best is kept for purposes of velocity, this is effectively an elitist algorithm.

There are no genetic operators (crossover or mutation).

This is a heuristic - convergence is highly likely, but not guaranteed. There is no good way to derive how close a solution is to the actual global maximum based on the algorithm performance. There may, of course, be other things known about the problem that can provide an upper bound for the difference.

The main upside of the algorithm is its simplicity. There are few tuning parameters and practically no restrictions on the fitness function (other than it being computable). Convergence is generally quite fast.

The downside is that the algorithm is very prone to getting stuck on local maximums, particularly if the gradient of the fitness function steep in the vicinity of the local extreme point. Once the particles have been pulled away from a global maximum due to faster convergence at a local maximum, there is nothing that pulls them back that direction.

Short break

The original plan was to finish up this semester and then take a two-week break. After this weekend, I'm revising that.

Yaya was out of town as her school band had a trip to Chicago. Kate and I took advantage of that, and some truly fine weather, to spend some quality down time together. Even though it was the shortest of breaks, it was surprisingly adequate to renew my spirits. So, I'm going shift right into paper-a-day mode tomorrow. I have three papers to review in prep for my last exam on Thursday, so this actually works out pretty well.

The way forward

While I take back none of my concerns, the right path forward is always just that: forward. It may well be that this endeavor is doomed but, if that turns out to be the case, it must be due to things outside of my control, not things that I simply gave up on.

With that said, here's the plan for the summer: read and comment on a paper every day (every day interpreted as 6 days a week minus a few vacation days). The only way I'm going to get a faculty member on board with this research is to show that it's viable. To do that, I need to complete my survey paper. To do that, I need to survey the literature.

It's not particularly inspiring (the "literature" is much more of a wasteland than one might hope). It's a lot of work. It's tedious at times. It involves a lot of effort wasted on papers that end up having no bearing on the problem. It's also what you do to prepare yourself for PhD research. I knew that going in; I had to do it for my master's work as well. I'll do it again. I just hope it works.

Last class ever?

Well, last graded class, perhaps. Today was the final class for both Set Theory and Evolutionary Algorithms. It's certainly conceivable that I'll take another class in an actual classroom for credit. But, most likely, I won't. All my remaining PhD credits will likely be directed research and I don't see why I'd take any other class for a grade.

Of course, it's not like I'll never set in a "class" again. One doesn't last long in my field without continually learning stuff. That often involves sitting in on a seminar or even a formal training course. Even as part of my journey at UMSL, I'll certainly attend lectures and colloquiums. But those aren't really the same thing as a true graded class.

I don't mind taking classes, but I surely won't miss exams. It's not that I'm bad at them; I just don't like them. Exams force you to put down bad answers. I don't like doing that. There are times in life when you have to make a call on the spot. But, unless we're talking trivial decisions, that's by far the exception rather than the rule. The common adult pattern is to take some time if you need it to get the answer right. Exams don't let you do that.  I'll be really happy next week to know that I'll probably not have to take an exam ever again.

I find your lack of faith disturbing...

I got a comment lately that my blog posts had gotten a bit dark. Well, I think they may be about to get a lot darker.

Simply put, I don't think anybody on the faculty at UMSL believes in me. I'm not sure why that's the case. I would think that a 4.0 student who's passed the Q and has two papers more or less ready for publication would generate some interest. But, that is definitely not the vibe I'm getting. One faculty member has more than subtly suggested that I may be at the wrong place.

I have no idea what to do in this situation. Frankly, I never even contemplated it. It's one thing to fight your own doubts; quite another to address those of others (especially when you don't even know where those doubts are coming from).

I'm not giving up yet but, if I can't find someone who is really serious about working with me on a thesis, I'll have to concede that this is a hopeless task.

That's what.

Ha, ha! Yeah, that whole I'm not sure what to do with myself in the evening didn't last long. My Set Theory prof changed his mind about the last homework. So, while I was planning on informally running through the exercises anyway, that meant I now had to actually do them right and typeset them. So, that took a while (but not as long as I feared it would; this last set wasn't nearly as rough as some of the earlier ones).

Anyway, it's here if you want to take a peek. I'll probably do a little revising, but not much. I want to turn it in Thursday and move on to finals prep.

Now what?

Well, I actually know the answer to that. Work is heating up big-time and the semester isn't completely done. I still have another Set Theory assignment and there's the little matter of finals. That said, it is almost surreal to have my term project out of the way. It's been so all consuming over the past few weeks that it just seems odd to not immediately start working on it after dinner.