Today's the first time I've heard of the term, from the news that Google acquired micro-blogging site Jaiku (a play on Haiku, by the way). I sort of like the idea, but don't intend on trying it. After all, most of my posts are fairly lengthy, like my last one, and I can't talk about integrating logical and stochastic artificial intelligence in 200 characters or less.
I like micro-blogging because of its spontaneousness. Instead, what I'll try to do is write down random thoughts in my Moleskine, and convert them into sentences, posts, essays later on.
That's what Moleskines are for.
Combining Logic and Stocastic Methods
Anonymous,
This is a computer science heavy post, so if you don't have the stomach for a lot of abstract, mathematical, theoretical ideas, you'd better skip this one.
I hope you would read it though.
Right now you can say that there are two big camps in artificial intelligence: those whose methods are based on logic, and those whose methods are based on mathematics.
One logic based system would be Cyc, a knowledge base which can reason. So if you put in "All men are mortals; Socrates is a man;" the program can tell you that "Socrates is a mortal." Well, not in plain English (although professor Chris Riesbeck and others are working on that), but it has an understanding of what those words mean. It connects, say, George Bush with USA, presidency, war on Iraq, and so on. Basically, it's the "common sense" that a lot of people feel computers don't have.
On the other hand are stochastic or statistical systems, generally labeled "machine learning" systems. They operate mostly on numbers, although some use strings and other data structures as well. Most of the cool-sounding AI techniques are in this category, including neural networks and genetic algorithms. These programs are essentially giant calculators, which take past events into account and try to mathematically create the optimal solution. For example, a photograph with a sun in it would have a much brighter spot somewhere on the surface, and the program could analyze the distribution of light over the surface to categorize whether a photo has the sun in it or not. It doesn't really know what a sun is (a star that gives off heat and light), but it does the job.
Obviously, both sides have their advantages and weaknesses, or there wouldn't be two camps. The logicians require large amounts of data beforehand - where the computer stores all that common knowledge - and it could take a very long time to logically derive anything. Using the Socrates example again, it also knows that men have two legs and dogs don't have four legs, so it might conclude that Socrates is not a dog, which is not very useful at all. The main problem here is how to sort through all this junk. For stochastic methods, the problem is the opposite. It is relatively faster than logic, but it doesn't really know much about the thing it's doing. You can say that there is no creativity involved, no way to do something unexpected. They are in a sense limited to what they were programmed to do, within a very specific domain.
So the idea I had, and have held it for a while now, is to introduce statistics into a knowledge base. This is how it will work. Cyc is essentially built on connections between ideas, a hierarchy of different relationships. Buying might be a subset of all possession transfers, which in turn might be a subset of all human interactions.
The problem is that each idea, object, and relationship has so much other stuff connected to it, you don't know where to begin. Example: a story about animals making false cries of danger might induce thoughts of Boy who Cried Wolf. There is however a lot of junk in both those stories, when the central concept is very abstract, that of deceiving others as entertainment and a trade off between humor and protection. Humans can find connections without a problem, but machines have to explore millions of concepts to hit on the right now.
So what you can do (or try do to; this is a theoretical discussion) is put a weight on each relationship. Wolves might be more connected to animals than to dogs, and birds more to doves than to penguins. Humans have a tendency to use shortcuts, cognitive heuristics so we don't just stand and think the whole day. Statistical information about the strength of each link would be similar, allowing the machine to know which links are more commonly encountered, and therefore should be explored first.
Taking knowledge bases as relationships between objects, the entire KB can be seen as a mathematical graph. And with statistical linkage associated with each link/edge, guess what? We now have a undirected weighted graph on our hands. The theorem proofer can now rely on the large amount of graph algorithms (Dijkstra's algorithm, A* search) to either find the shortest path to proof a statement, or to generate new statements. There is still the matter of the machine not knowing what results are important, but that's a different problem.
One last thing about the statistics: one way to get the different weights between different (but related) objects would to be search another knowledge base: Wikipedia. Although Wikipedia is written in English, the text is still somewhat machine readable. For example, the machine will know to link to other pages if the user encloses something in double brackets ([[ ]]). So one easy way to find the links is take the inverse of the number of links it takes to link two wikipedia pages. This is in fact another graph traversal problem, but much more easily solved. So objects which are link directly to each other (say dog and animal) would have a much higher chance of being evaluated than objects far apart (say dog and black holes).
I think that's one way to take advantage of both worlds. I don't know if this has been done before, and I'm not really in a mood to find out.
I hope you would read it though.
Right now you can say that there are two big camps in artificial intelligence: those whose methods are based on logic, and those whose methods are based on mathematics.
One logic based system would be Cyc, a knowledge base which can reason. So if you put in "All men are mortals; Socrates is a man;" the program can tell you that "Socrates is a mortal." Well, not in plain English (although professor Chris Riesbeck and others are working on that), but it has an understanding of what those words mean. It connects, say, George Bush with USA, presidency, war on Iraq, and so on. Basically, it's the "common sense" that a lot of people feel computers don't have.
On the other hand are stochastic or statistical systems, generally labeled "machine learning" systems. They operate mostly on numbers, although some use strings and other data structures as well. Most of the cool-sounding AI techniques are in this category, including neural networks and genetic algorithms. These programs are essentially giant calculators, which take past events into account and try to mathematically create the optimal solution. For example, a photograph with a sun in it would have a much brighter spot somewhere on the surface, and the program could analyze the distribution of light over the surface to categorize whether a photo has the sun in it or not. It doesn't really know what a sun is (a star that gives off heat and light), but it does the job.
Obviously, both sides have their advantages and weaknesses, or there wouldn't be two camps. The logicians require large amounts of data beforehand - where the computer stores all that common knowledge - and it could take a very long time to logically derive anything. Using the Socrates example again, it also knows that men have two legs and dogs don't have four legs, so it might conclude that Socrates is not a dog, which is not very useful at all. The main problem here is how to sort through all this junk. For stochastic methods, the problem is the opposite. It is relatively faster than logic, but it doesn't really know much about the thing it's doing. You can say that there is no creativity involved, no way to do something unexpected. They are in a sense limited to what they were programmed to do, within a very specific domain.
So the idea I had, and have held it for a while now, is to introduce statistics into a knowledge base. This is how it will work. Cyc is essentially built on connections between ideas, a hierarchy of different relationships. Buying might be a subset of all possession transfers, which in turn might be a subset of all human interactions.
The problem is that each idea, object, and relationship has so much other stuff connected to it, you don't know where to begin. Example: a story about animals making false cries of danger might induce thoughts of Boy who Cried Wolf. There is however a lot of junk in both those stories, when the central concept is very abstract, that of deceiving others as entertainment and a trade off between humor and protection. Humans can find connections without a problem, but machines have to explore millions of concepts to hit on the right now.
So what you can do (or try do to; this is a theoretical discussion) is put a weight on each relationship. Wolves might be more connected to animals than to dogs, and birds more to doves than to penguins. Humans have a tendency to use shortcuts, cognitive heuristics so we don't just stand and think the whole day. Statistical information about the strength of each link would be similar, allowing the machine to know which links are more commonly encountered, and therefore should be explored first.
Taking knowledge bases as relationships between objects, the entire KB can be seen as a mathematical graph. And with statistical linkage associated with each link/edge, guess what? We now have a undirected weighted graph on our hands. The theorem proofer can now rely on the large amount of graph algorithms (Dijkstra's algorithm, A* search) to either find the shortest path to proof a statement, or to generate new statements. There is still the matter of the machine not knowing what results are important, but that's a different problem.
One last thing about the statistics: one way to get the different weights between different (but related) objects would to be search another knowledge base: Wikipedia. Although Wikipedia is written in English, the text is still somewhat machine readable. For example, the machine will know to link to other pages if the user encloses something in double brackets ([[ ]]). So one easy way to find the links is take the inverse of the number of links it takes to link two wikipedia pages. This is in fact another graph traversal problem, but much more easily solved. So objects which are link directly to each other (say dog and animal) would have a much higher chance of being evaluated than objects far apart (say dog and black holes).
I think that's one way to take advantage of both worlds. I don't know if this has been done before, and I'm not really in a mood to find out.
Quick Updates
Anonymous,
I'm slipping. Here's what has happened in my life since the last post:
- Saw Ratatouille. (Walked out of it. Did not buy it. Although, it was better than that "Throw Mama from the Tambien" crap.)
- I bought a bike! A Trek 750. Haven't gone on a trip yet though. Should do so sometime this week.
- Wrote a chat bot which insults people. Here's a transcript:
hello
(SHUT UP AND STOP ACTING POLITE)
no you shut up
(SHUT IT YOURSELF)
it was your faut
(LIKE I WILL EVER IT WAS MY FAUT)
Yes it was
(IN YOUR REAR COW DERRIERE)
dont insult me
(YOU ARTLESS TOAD-SPOTTED HORN-BEAST!)
you pinhead
(NO YOURE PINHEAD)
Mother lover
(IN YOUR REAR COW DERRIERE)
This is stupid
(IN YOUR REAR COW DERRIERE)
youre so dumb you have to repeat yourself
(NO YOURE SO DUMB YOU HAVE TO REPEAT YOURSELF)
I didnt repeat myself
(THAT ALL YOU GOT LOSER)
Im not a loser
(TO CALL YOU STUPID WOULD BE AN INSULT TO STUPID PEOPLE)
alright Im sorry
(YOUD BETTER BE SORRY)
I give up
(GO HOME LOSER)
That's it for now!
- Saw Ratatouille. (Walked out of it. Did not buy it. Although, it was better than that "Throw Mama from the Tambien" crap.)
- I bought a bike! A Trek 750. Haven't gone on a trip yet though. Should do so sometime this week.
- Wrote a chat bot which insults people. Here's a transcript:
hello
(SHUT UP AND STOP ACTING POLITE)
no you shut up
(SHUT IT YOURSELF)
it was your faut
(LIKE I WILL EVER IT WAS MY FAUT)
Yes it was
(IN YOUR REAR COW DERRIERE)
dont insult me
(YOU ARTLESS TOAD-SPOTTED HORN-BEAST!)
you pinhead
(NO YOURE PINHEAD)
Mother lover
(IN YOUR REAR COW DERRIERE)
This is stupid
(IN YOUR REAR COW DERRIERE)
youre so dumb you have to repeat yourself
(NO YOURE SO DUMB YOU HAVE TO REPEAT YOURSELF)
I didnt repeat myself
(THAT ALL YOU GOT LOSER)
Im not a loser
(TO CALL YOU STUPID WOULD BE AN INSULT TO STUPID PEOPLE)
alright Im sorry
(YOUD BETTER BE SORRY)
I give up
(GO HOME LOSER)
That's it for now!
Fire Alarms
Anonymous, 2007-10-05
I think it's normal for some idiot college student to not know how to use a microwave, set popcorn in there for 60 minutes, and cause the fire alarm to go off. I think it's normal for that to happen at 3 in the morning, since that's when students are most active. They do it once, they know how much it annoys other people, they learn what they did wrong, they don't do it again.
That's one alarm.
This morning our fire alarm went off. Three times. Within 3 hours.
Either that's a really dumb (and really persistent student, or the system's messed up.
I really don't know which one to go with.
That's one alarm.
This morning our fire alarm went off. Three times. Within 3 hours.
Either that's a really dumb (and really persistent student, or the system's messed up.
I really don't know which one to go with.
Realizations from Optical Illusions
Anonymous,
That's a mouthful. We were doing some sensory (mostly optical) illusions today in cognitive psych, and while there were a number which were new to me, a bunch of them were pretty familiar ones. I thought back to my childhood when I bought several optical illusion books, and came to a realization:
I enjoy being tricked by my brain.
First, let me give you these two lesser known (but really good) optical illusions:
The Spinning Ballerina Illusion
The Face Mask Illusion
The second one in particular shows how our brain is so used to seeing faces, that we can't help but believe the face pops up, even when we know it's the back of a mask. Of course, we all know that our senses can't be trusted entirely. Not only our sight, but also our sense of touch (poke yourself with two toothpicks about a quarter inch apart; you could tell the difference with your fingers but not your back). Our brain is trying to make sense of a lot of information at the same time, and takes some short cuts in doing so, but in doing that some very specific situations results in our brain telling us the wrong thing.
You can argue that there are other areas in life where our brain makes mistakes. One area I thought of are jokes, especially puns. Puns work by defying our common expectations, which may be cultural or habitual in origin, but this doesn't show up in the grammatical structure of the sentence itself. Here are a few puns, although I must warn you that I take Edgar Allan Poe's quotation ("The goodness of the true pun is in the direct ratio of its intolerability") to heart.
Of course, my love for both of these things doesn't really indicate that I like being tricked by my brain, but that I like finding how the brain works.
Some other area which this applies to came into my mind moments before, but now it's gone. Isn't that fascinating?
I enjoy being tricked by my brain.
First, let me give you these two lesser known (but really good) optical illusions:
The Spinning Ballerina Illusion
The Face Mask Illusion
The second one in particular shows how our brain is so used to seeing faces, that we can't help but believe the face pops up, even when we know it's the back of a mask. Of course, we all know that our senses can't be trusted entirely. Not only our sight, but also our sense of touch (poke yourself with two toothpicks about a quarter inch apart; you could tell the difference with your fingers but not your back). Our brain is trying to make sense of a lot of information at the same time, and takes some short cuts in doing so, but in doing that some very specific situations results in our brain telling us the wrong thing.
You can argue that there are other areas in life where our brain makes mistakes. One area I thought of are jokes, especially puns. Puns work by defying our common expectations, which may be cultural or habitual in origin, but this doesn't show up in the grammatical structure of the sentence itself. Here are a few puns, although I must warn you that I take Edgar Allan Poe's quotation ("The goodness of the true pun is in the direct ratio of its intolerability") to heart.
- Two antennas met on a roof, fell in love and got married. The ceremony wasn't much, but the reception was excellent.
- Deja Moo: The feeling that you've heard this bull before.
- Here's a dead give-away: what's a will?
Of course, my love for both of these things doesn't really indicate that I like being tricked by my brain, but that I like finding how the brain works.
Some other area which this applies to came into my mind moments before, but now it's gone. Isn't that fascinating?
VOIP
Anonymous, 2007-10-03
As many of you know (how many of you are there anyway?), I don't like cell phones. It's mostly because I don't like to be interrupted so often, and just let anyone find me. Plus, I already have a ton of stuff in my pockets (Moleskine, pen, keys, coins, bills), and I don't want to cram them even more. Some people have said that by doing so I inconvenient people, which is sort of true. Some people have also proposed to give me a phone for my birthday. Er, thanks.
You see, it's not that I don't think phones are useful, I just don't like them.
But my solution is actually not too bad of one. At Northwestern each dorm room gets their own phone line, but while we can receive calls, we have to pay to call out. Works okay if people really want to find me, but email is still easier, as I can check that anywhere on a computer and I don't have an answering machine. In fact, despite living within 15 minutes walking distance from all of campus, I like to check my email whereever I am first before going back to my dorm. So people can find me (of sorts) if they want to, but that does not help me find other people, especially for people who don't check their email very often.
The solution: Click2Voice.com
Click2Voice is a free VOIP page. It's not software that you install, but you use their service through their page, which influences the real world. You enter your phone number, and the number you're going to call, and hit enter. In 5 seconds, your phone will ring, as though someone is calling you. You pick up, and you will hear a prerecorded message thanking you for using their service and to hit 1 to connect your call. You do so, and in another 10 seconds your calls goes through, just as though you had dialed that number.
The beauty of this website is that it doesn't require your phone to be able to call out, which is perfect for my dorm line. So now I can find people, people can find me with less difficulty, and I don't have to put extra crap (yes I did just call cell phones "crap) in my pockets.
Perfect.
Eventually though, I'm thinking I'll get a good headset for my computer, sign up for Skype Out, and do everything through my laptop instead. Look, ma, no phones at all!
You see, it's not that I don't think phones are useful, I just don't like them.
But my solution is actually not too bad of one. At Northwestern each dorm room gets their own phone line, but while we can receive calls, we have to pay to call out. Works okay if people really want to find me, but email is still easier, as I can check that anywhere on a computer and I don't have an answering machine. In fact, despite living within 15 minutes walking distance from all of campus, I like to check my email whereever I am first before going back to my dorm. So people can find me (of sorts) if they want to, but that does not help me find other people, especially for people who don't check their email very often.
The solution: Click2Voice.com
Click2Voice is a free VOIP page. It's not software that you install, but you use their service through their page, which influences the real world. You enter your phone number, and the number you're going to call, and hit enter. In 5 seconds, your phone will ring, as though someone is calling you. You pick up, and you will hear a prerecorded message thanking you for using their service and to hit 1 to connect your call. You do so, and in another 10 seconds your calls goes through, just as though you had dialed that number.
The beauty of this website is that it doesn't require your phone to be able to call out, which is perfect for my dorm line. So now I can find people, people can find me with less difficulty, and I don't have to put extra crap (yes I did just call cell phones "crap) in my pockets.
Perfect.
Eventually though, I'm thinking I'll get a good headset for my computer, sign up for Skype Out, and do everything through my laptop instead. Look, ma, no phones at all!
Lasers
Anonymous, 2007-10-02
Evanston is foggy tonight. Visibility is about 20 feet, max. It was a slightly overcast day, but I didn't expect fog to roll in. Except it did, after I left GSW.
And one cool thing about fog: each particle of water reflects light. So what I did, I took out my laser pointer. Usually, laser pointers have invisible beams, because there isn't enough energy or something. There are ones you can get with a visible beam, on ThinkGeek for example, and I think they use four times more energy than normal pointers. Well, one thing this fog is good at: reflecting light. Whenever my laser points into the fog, I can see the entire beam. Now that is cool.
And one cool thing about fog: each particle of water reflects light. So what I did, I took out my laser pointer. Usually, laser pointers have invisible beams, because there isn't enough energy or something. There are ones you can get with a visible beam, on ThinkGeek for example, and I think they use four times more energy than normal pointers. Well, one thing this fog is good at: reflecting light. Whenever my laser points into the fog, I can see the entire beam. Now that is cool.
Subscribe to:
Posts
(
Atom
)