home
on exploration, introspection and creation

Archive for March, 2010

Evolution, genetic algorithms, and why AI may continue to suck

Wednesday, March 24th, 2010

I talked a lot about evolution before; I find it fascinating. There are many lessons to be drawn from the study of evolution as an epiphenomenon (i.e. I don’t want to study its effects; I want to study its emergence). Here is one interesting angle.

Photosynthesis is one of the most interesting forms of harnessing the energy of the Sun. There are good reasons why having such an efficient mechanism is beneficial to organisms (curiously, the animal kingdom has foregone this mechanism as it’s mostly only applicable to stationary organisms, or organisms with certain energy needs (or something else; I admit that I simply don’t know enough biology to be able to answer that)) — the organism that has evolved an ability to gather the energy from the sun is much less dependent on volatile scarce resources and instead can use what is essentially an infinite source of energy.

Similarly, fireflies have evolved the ability to emit light. The mechanism that makes it possible is a very efficient way of emitting light (if you ever touched one in flight I’m sure you were shocked at how unwarm it was).

It seems that evolution happens to come up with some pretty interesting solutions to problems. So why not use evolution to solve our problems for us, i.e. create artificial conditions where survival necessitates a development of a specific mechanism. For example, if we wanted Nature to help us come up with walls that withstand bomb blasts, we would create extremely harsh conditions that simulate bomb blasts (for example, create a chamber featuring extremely strong winds and extremely frequent earthquake-like conditions) and see what evolution does for us while trying to maximize the changes of survival of organisms that have the unfortunate luck of living in such a chamber.

If you studied computer science, you will no doubt tell me that we already do something like that — genetic algorithms are essentially a way to come up with solutions to problems by letting the strongest algorithms survive. This is true, but one crucial difference is that natural evolution has at its disposal two things that genetic algorithms don’t — the complexity of organisms already in existence, and the complexity of laws of physics. In other words, the reason evolution comes up with very impressive results is that it’s essentially an incredibly powerful computational system that operates on extremely complex systems (both the organisms and the conditions). Evolution comes up with designs only as strong as the conditions it operates under, or, in other words, the information content of the designs that are a result of evolution is significantly smaller than the information content of the models that evolution operates on. In the case of genetic algorithms, such models are usually fairly simple and in general, it’s not particularly useful to spend a lot of effort constructing an elaborate world for genetic algorithms to roam in–usually computer scientists use their own intelligence and solve the problem with pen and paper.

For example, it turns out that photosynthesis is probably not possible without quantum effects (this is by itself quite a breathtaking thing, if it’s true). If I wanted to simulate a world in which the “algorithm” of photosynthesis would appear through evolution, I would have to ensure that the world my algorithms live in is as complex as our current knowledge of physics (and even that might not be enough!), as well as ensure that the “organisms” are complex enough so as to take advantage of photosynthesis.

This indirectly points to my belief that we are far away from any semblance of actual artificial intelligence (like one featured in countless movies). Intelligence evolved in organisms which were incredibly complex, in a world that was just as complex. It’s likely (though not certain — after all, evolution is not the most efficient problem-solver) that intelligence has a very large information content.

So we probably can’t emulate evolution effectively. But we can still take advantage of it, since many of mankind’s inventions are influenced by what we have observed in the natural world. One big problem to running any kind of experiments like this is that evolution is painfully slow: it takes thousands of generations to see any meaningful change. I think we can get around this in three ways:

  • Making it desirable in our constructed universe to a generation to be short-lived — that way mutations will happen more frequently
  • Parallelizing the problem by crossing populations or taking advantage of horizontal evolution — that way it will take fewer generations in the critical path of change that generates our desired outcome
  • (This one I’m less bullish on) Increasing the probability of mutations, through the introduction of radioactivity — that way mutations will happen faster, so the critical path of change will involve fewer generations

In the extreme, we are limited by the speed of cell division (because that’s when mutations happen) which becomes the clock speed of our genetic computers.

On a Limitation of (Some) Time Travel

Wednesday, March 24th, 2010

Let’s be honest, time travel fascinates us all, no matter how much physics we know. There is something mystical to me about how elusive the answer to whether it’s possible is.

There are many theories of time travel, those admitting the possibility of the Grandfather Paradox and those that preclude it; those that attempt to explain why nobody from the future visited us yet; those that involve Nature as some kind of intelligent designer that scrambles to avoid paradoxes (I’m looking at you, J. J. Abrams); those that constrain time travel to a subset of circumstances (my favorite is the theory of time travel put forth in Primer).

I considered the other day what consequences unrestrained travel back in time would have. I can’t just appear without causing any side effects on the time-space that I traveled to: the mass/energy of the universe wouldn’t be conserved (so I could build a kind of automated time travel machine that continuously adds energy or mass to the universe, or–even better–reduce the entropy of the entire universe). So my arrival needs to be coupled with disappearance of energy or mass. Where would that energy disappear to?

A Book on the Obvious Things

Wednesday, March 24th, 2010

For a while I’ve been wanting to write a book about things that seem to me like they should be common knowledge. “Things I Know that I Think You Should Know”, or something like that.

Then I thought about it some more. While I think the book would still be fun, it wouldn’t be all that effective: people either know these things, or they don’t. If they do, they won’t need the book. If they don’t know the things that seem obvious to me, they probably won’t find much use for such a book.

You don’t know you need it until you have it

Thursday, March 11th, 2010

(I promised my friend I would write this post some day–here you are, buddy.)

A great way to achieve goals is by enumerating problems first and then solving them. However, this method has a blind spot — value can also be added not by solving pain but by making what is good better.

A good manifestation of this is the “you don’t know you need it until you have it” phenomenon. My friend and I would spend hours discussing why I thought he should get a touch phone. “I don’t perceive any problems with my life that a touch phone could solve,” he would say. Well, I used to think that too at the end of my Freshman year in college, as all my friends tried to convince me to buy a cell phone.

So–for the benefit of my friend–here are the things that having a touch phone (iPhone specifically but the argument can easily be used for any touch phones) enabled me to do that I never even thought of being able to do

  • I can figure out how to get from Grand Central to Columbia University by subway. Previously I either had to take a map with me, look for a map at the station, or get on a train that was hopefully the right one and look through the map there. I can even tell how long the trip’s going to take.
  • I can figure out where that restaurant at which I was supposed to meet my friend was–I only remember the name. Previously I would have to call someone or ask and hope they would know.
  • I can mark where I parked my car so I don’t have to worry that I’ll forget the day after when I have to pick it up.
  • I know when the train leaves Grand Central so I can spend more time hanging out with my friends rather than stranded at the train station.
  • I can very quickly resolve these bets that I often make with others, related to some particular factoid that we disagreed on at dinner.
  • I will never, ever forget anything because I can write a note to myself at any point.
  • I will never, ever be bored because I can
    • Listen to radio
    • Play games
    • Listen to audiobooks
    • Read books
    • Listen to music
    • Write a post in my blog
  • I can pretend to be a good cook because I can pull up a recipe anytime
  • I can recommend a restaurant to a group of people based on their recommendations

I’m just going to stop here because, quite frankly, it’s like listing the benefits of the Internet (the “touch” part is incredibly useful because it makes user experience bearable — doing most of the above on a WAP-enabled phone or even a real browser but with a keypad-activated cursor is a pain in the neck).

Of course, one drawback of making good things better is that your expectations are raised. I remember thinking a month after I got my iPhone, “This is the most user-friendly, useful device I have ever used. My life is so much easier and richer right now.”. Now, two years in, I am very frustrated with it. It’s unresponsive, the lack of Flash annoys the hell out of me, I have to spend so long looking up directions. In fact, I’m just a much more discerning customer.

The Encyclopedia-then-Internet generation

Monday, March 8th, 2010

I remember the time, when I was little, when there was no Internet. All my generalist knowledge about the world came from two sources: my parents, and an encyclopedia we had at the house. In a way, the encyclopedia was like the Internet (combined with Yellow Pages). If I needed to draw a picture of a bird, I browsed the encyclopedia for images of birds. If I was bored, I’d open the encyclopedia on a random page.

I think that spending all of elementary school and a part of middle school like that made me appreciate information. I think it is the case that my generation pays closer attention to the quality of information, intuitively comparing what they see to what they would see in an encyclopedia. And I think we are better at finding the more obscure information (since search was linear–no index–I had to be clever about what I looked for). But, on the other hand, we’re probably struggling more with the amount of information (unless we’ve stayed on top of it by following the trend for all these years–still, if I really want to be up to speed I feel the need to read about 200 blog posts every day while back in the day it was a few pages in a newspaper).

On Motivation (part II)

Monday, March 8th, 2010

I continue to be amazed by what motivates me. One day I’d just wake up and want to get stuff done. On another, seemingly identical, occasion, I would feel a deep lack of desire to do anything.

Over the past month I’ve had several discoveries. While I know that they are likely different for different people, I encourage everyone to think about what motivates them. Often the factors seem completely arbitrary and are unlike what many self-help publications purport to convince us of.

  • Sometimes I need to be lazy so that in the medium term I can get stuff done. This, I think, is the most important thing that people overlook. I think that this is partly due to the need for my brain to relax (or stop concentrating on one area such as the area responsible for planning and achievement), and partly due to the fact that the desire to get things done is a function of our overall state of mind: spending some time being lazy makes me more motivated to get stuff done later (the “broccoli and ice cream” effect)
  • The most surprising of all was the discovery that if I get home from work before 7pm, I am significantly more motivated to get “life” work done. I haven’t figured out why that is; but this rule seems to work very well for me
  • Several small successes in a row incentivize me to get more done
  • What reduces my anxiety the most is not (in general) getting things done but having a good grasp of what needs to be done and the priority of these things. One thing I therefore tried to do was not organizing my todo list but instead just knocking things off the list so I don’t become complacent having a well-formed todo list

The caveats of logical thinking (part II)

Monday, March 8th, 2010

The problem with relying on logical thinking alone is that it can let your guard down.

In the beginning of the twentieth century, mathematicians realized that logic doesn’t actually get you that far. As described previously, logic is as strong as a set of axioms behind it. This is fine in a highly contrived world, like one of Peano or Euclid but in the real world, there are simply too many axiom sets.

Roughly, the axioms correspond to two things:

  • One’s philosophy and values, fundamental things one can’t break down any further
  • Information that one has to treat as fundamental simply because breaking it down is too hard

To the sixteenth-century Europeans, for example, the properties of magnets were axiomatic because, even to the respected scientists of that time, the knowledge necessary to understand magnetism as a consequence of some simpler laws was missing and could not easily be derived from first principles (i.e. it’s not that the scientists at the time were illogical).

If you are talking to somebody who is a good logical thinker and also has good argumentative skills, it’s possible that he or she may convince you pretty much of everything by selecting the axioms he or she cares about and using them in the argument. You would be able to follow the entire conversation without ever exposing a logical flaw in the argument but what you may fail to see is that at the end of the day, there is a degree of arbitrariness in the argument that comes from what axioms were picked. In other words, smart people can rationalize anything.

What should you do when this happens? It’s not easy.

  • Understand that there are other axioms out there. Perhaps there are other reasons against something. Since they are axioms, it will be very hard to trade them off against each other. Point out the fact that neither you nor the person you are talking to are sufficiently equipped to perform the trade-off analysis and thus either of you may be making a wrong decision.
  • Use other sources of reasoning other than logic. Intuition and experience is a good one — it shouldn’t be relied upon too heavily but you can extract value in the fact that you saw something fail several dozen times: perhaps it’s more prone to failure, regardless of whether you can derive the reason using only logic. Sometimes decisions hinge on human nature — your understanding of human emotion can help you make a better decision.

In other words, prefer logical thinking, but don’t use it as a silver bullet.